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PREFACE 


Here is your book, the one your thousands of letters have asked us 
to publish. It has taken us years to do, checking and rechecking countless 
recipes to bring you only the best, only the interesting, only the perfect. 
Now we can say, without a shadow of a doubt, that every single one of them, 
if you follow the directions to the letter, will work for you exactly as well 
as it did for us, even if you have never cooked before. 

— McCall's Cookbook (1963) 


The PROCESS of preparing programs for a digital computer is especially attrac- 
tive, not only because it can be economically and scientifically rewarding, but 
also because it can be an aesthetic experience much like composing poetry or 
music. This book is the first volume of a multi-volume set of books that has been 
designed to train the reader in various skills that go into a programmer’s craft. 

The following chapters are not meant to serve as an introduction to computer 
programming; the reader is supposed to have had some previous experience. The 
prerequisites are actually very simple, but a beginner requires time and practice 
in order to understand the concept of a digital computer. The reader should 
possess: 

a) Some idea of how a stored-program digital computer works; not necessarily 
the electronics, rather the manner in which instructions can be kept in the 
machine’s memory and successively executed. 

b) An ability to put the solutions to problems into such explicit terms that a 
computer can “understand” them. (These machines have no common sense; 
they do exactly as they are told, no more and no less. This fact is the 
hardest concept to grasp when one first tries to use a computer.) 

c) Some knowledge of the most elementary computer techniques, such as loop- 
ing (performing a set of instructions repeatedly), the use of subroutines, and 
the use of indexed variables. 

d) A little knowledge of common computer jargon — “memory,” “registers,” 
“bits,” “floating point,” “overflow,” “software.” Most words not defined in 
the text are given brief definitions in the index at the close of each volume. 

These four prerequisites can perhaps be summed up into the single requirement 
that the reader should have already written and tested at least, say, four pro- 
grams for at least one computer. 

I have tried to write this set of books in such a way that it will fill several 
needs. In the first place, these books are reference works that summarize the 
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knowledge that has been acquired in several important fields. In the second place, 
they can be used as textbooks for self-study or for college courses in the computer 
and information sciences. To meet both of these objectives, I have incorporated 
a large number of exercises into the text and have furnished answers for most 
of them. I have also made an effort to fill the pages with facts rather than with 
vague, general commentary. 

This set of books is intended for people who will be more than just casually 
interested in computers, yet it is by no means only for the computer specialist. 
Indeed, one of my main goals has been to make these programming techniques 
more accessible to the many people working in other fields who can make fruitful 
use of computers, yet who cannot afford the time to locate all of the necessary 
information that is buried in technical journals. 

We might call the subject of these books “nonnumerical analysis.” Comput- 
ers have traditionally been associated with the solution of numerical problems 
such as the calculation of the roots of an equation, numerical interpolation 
and integration, etc., but such topics are not treated here except in passing. 
Numerical computer programming is an extremely interesting and rapidly ex- 
panding field, and many books have been written about it. Since the early 
1960s, however, computers have been used even more often for problems in which 
numbers occur only by coincidence; the computer’s decision-making capabilities 
are being used, rather than its ability to do arithmetic. We have some use 
for addition and subtraction in nonnumerical problems, but we rarely feel any 
need for multiplication and division. Of course, even a person who is primarily 
concerned with numerical computer programming will benefit from a study of 
the nonnumerical techniques, for they are present in the background of numerical 
programs as well. 

The results of research in nonnumerical analysis are scattered throughout 
numerous technical journals. My approach has been to try to distill this vast 
literature by studying the techniques that are most basic, in the sense that they 
can be applied to many types of programming situations. I have attempted to 
coordinate the ideas into more or less of a “theory,” as well as to show how the 
theory applies to a wide variety of practical problems. 

Of course, “nonnumerical analysis” is a terribly negative name for this field 
of study; it is much better to have a positive, descriptive term that characterizes 
the subject. “Information processing” is too broad a designation for the material 
I am considering, and “programming techniques” is too narrow. Therefore I wish 
to propose analysis of algorithms as an appropriate name for the subject matter 
covered in these books. This name is meant to imply “the theory of the properties 
of particular computer algorithms.” 

The complete set of books, entitled The Art of Computer Programming, has 
the following general outline: 

Volume 1. Fundamental Algorithms 
Chapter 1. Basic Concepts 
Chapter 2. Information Structures 
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Volume 2. Seminumerical Algorithms 

Chapter 3. Random Numbers 
Chapter 4. Arithmetic 

Volume 3. Sorting and Searching 

Chapter 5. Sorting 
Chapter 6. Searching 

Volume 4. Combinatorial Algorithms 

Chapter 7. Combinatorial Searching 
Chapter 8. Recursion 

Volume 5. Syntactical Algorithms 

Chapter 9. Lexical Scanning 
Chapter 10. Parsing 

Volume 4 deals with such a large topic, it actually represents three separate books 
(Volumes 4A, 4B, and 4C). Two additional volumes on more specialized topics 
are also planned: Volume 6, The Theory of Languages (Chapter 11); Volume 7, 
Compilers (Chapter 12). 

I started out in 1962 to write a single book with this sequence of chapters, 
but I soon found that it was more important to treat the subjects in depth rather 
than to skim over them lightly. The resulting length of the text has meant that 
each chapter by itself contains more than enough material for a one-semester 
college course; so it has become sensible to publish the series in separate volumes. 
I know that it is strange to have only one or two chapters in an entire book, but 
I have decided to retain the original chapter numbering in order to facilitate 
cross-references. A shorter version of Volumes 1 through 5 is planned, intended 
specifically to serve as a more general reference and/or text for undergraduate 
computer courses; its contents will be a subset of the material in these books, 
with the more specialized information omitted. The same chapter numbering 
will be used in the abridged edition as in the complete work. 

The present volume may be considered as the “intersection” of the entire set, 
in the sense that it contains basic material that is used in all the other books. 
Volumes 2 through 5, on the other hand, may be read independently of each 
other. Volume 1 is not only a reference book to be used in connection with the 
remaining volumes; it may also be used in college courses or for self-study as a 
text on the subject of data structures (emphasizing the material of Chapter 2), 
or as a text on the subject of discrete mathematics (emphasizing the material 
of Sections 1.1, 1.2, 1.3.3, and 2.3.4), or as a text on the subject of machine- 
language programming (emphasizing the material of Sections 1.3 and 1.4). 

The point of view I have adopted while writing these chapters differs from 
that taken in most contemporary books about computer programming in that 
I am not trying to teach the reader how to use somebody else’s software. I am 
concerned rather with teaching people how to write better software themselves. 
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My original goal was to bring readers to the frontiers of knowledge in every 
subject that was treated. But it is extremely difficult to keep up with a field 
that is economically profitable, and the rapid rise of computer science has made 
such a dream impossible. The subject has become a vast tapestry with tens of 
thousands of subtle results contributed by tens of thousands of talented people 
all over the world. Therefore my new goal has been to concentrate on “classic” 
techniques that are likely to remain important for many more decades, and to 
describe them as well as I can. In particular, I have tried to trace the history 
of each subject, and to provide a solid foundation for future progress. I have 
attempted to choose terminology that is concise and consistent with current 
usage. I have tried to include all of the known ideas about sequential computer 
programming that are both beautiful and easy to state. 

A few words are in order about the mathematical content of this set of books. 
The material has been organized so that persons with no more than a knowledge 
of high-school algebra may read it, skimming briefly over the more mathematical 
portions; yet a reader who is mathematically inclined will learn about many 
interesting mathematical techniques related to discrete mathematics. This dual 
level of presentation has been achieved in part by assigning ratings to each of the 
exercises so that the primarily mathematical ones are marked specifically as such, 
and also by arranging most sections so that the main mathematical results are 
stated before their proofs. The proofs are either left as exercises (with answers 
to be found in a separate section) or they are given at the end of a section. 

A reader who is interested primarily in programming rather than in the 
associated mathematics may stop reading most sections as soon as the math- 
ematics becomes recognizably difficult. On the other hand, a mathematically 
oriented reader will find a wealth of interesting material collected here. Much of 
the published mathematics about computer programming has been faulty, and 
one of the purposes of this book is to instruct readers in proper mathematical 
approaches to this subject. Since I profess to be a mathematician, it is my duty 
to maintain mathematical integrity as well as I can. 

A knowledge of elementary calculus will suffice for most of the mathematics 
in these books, since most of the other theory that is needed is developed herein. 
However, I do need to use deeper theorems of complex variable theory, probability 
theory, number theory, etc., at times, and in such cases I refer to appropriate 
textbooks where those subjects are developed. 

The hardest decision that I had to make while preparing these books con- 
cerned the manner in which to present the various techniques. The advantages of 
flow charts and of an informal step-by-step description of an algorithm are well 
known; for a discussion of this, see the article “Computer-Drawn Flowcharts” 
in the ACM Communications , Vol. 6 (September 1963), pages 555-563. Yet a 
formal, precise language is also necessary to specify any computer algorithm, 
and I needed to decide whether to use an algebraic language, such as ALGOL 
or FORTRAN, or to use a machine-oriented language for this purpose. Per- 
haps many of today’s computer experts will disagree with my decision to use a 
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machine-oriented language, but I have become convinced that it was definitely 
the correct choice, for the following reasons: 

a) A programmer is greatly influenced by the language in which programs are 
written; there is an overwhelming tendency to prefer constructions that are 
simplest in that language, rather than those that are best for the machine. 
By understanding a machine-oriented language, the programmer will tend 
to use a much more efficient method; it is much closer to reality. 

b) The programs we require are, with a few exceptions, all rather short, so with 
a suitable computer there will be no trouble understanding the programs. 

c) High-level languages are inadequate for discussing important low-level de- 
tails such as coroutine linkage, random number generation, multi-precision 
arithmetic, and many problems involving the efficient usage of memory. 

d) A person who is more than casually interested in computers should be well 
schooled in machine language, since it is a fundamental part of a computer. 

e) Some machine language would be necessary anyway as output of the software 
programs described in many of the examples. 

f) New algebraic languages go in and out of fashion every five years or so, while 
I am trying to emphasize concepts that are timeless. 

From the other point of view, I admit that it is somewhat easier to write programs 
in higher-level programming languages, and it is considerably easier to debug 
the programs. Indeed, I have rarely used low-level machine language for my 
own programs since 1970, now that computers are so large and so fast. Many 
of the problems of interest to us in this book, however, are those for which 
the programmer’s art is most important. For example, some combinatorial 
calculations need to be repeated a trillion times, and we save about 11.6 days 
of computation for every microsecond we can squeeze out of their inner loop. 
Similarly, it is worthwhile to put an additional effort into the writing of software 
that will be used many times each day in many computer installations, since the 
software needs to be written only once. 

Given the decision to use a machine-oriented language, which language 
should be used? I could have chosen the language of a particular machine X, 
but then those people who do not possess machine X would think this book is 
only for A-people. Furthermore, machine X probably has a lot of idiosyncrasies 
that are completely irrelevant to the material in this book yet which must be 
explained; and in two years the manufacturer of machine X will put out machine 
X -)- 1 or machine 10A, and machine X will no longer be of interest to anyone. 

To avoid this dilemma, I have attempted to design an “ideal” computer 
with very simple rules of operation (requiring, say, only an hour to learn) , which 
also resembles actual machines very closely. There is no reason why a student 
should be afraid of learning the characteristics of more than one computer; once 
one machine language has been mastered, others are easily assimilated. Indeed, 
serious programmers may expect to meet many different machine languages in 
the course of their careers. So the only remaining disadvantage of a mythical 
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machine is the difficulty of executing any programs written for it. Fortunately, 
that is not really a problem, because many volunteers have come forward to 
write simulators for the hypothetical machine. Such simulators are ideal for 
instructional purposes, since they are even easier to use than a real computer 
would be. 

I have attempted to cite the best early papers in each subject, together with 
a sampling of more recent work. When referring to the literature, I use standard 
abbreviations for the names of periodicals, except that the most commonly cited 
journals are abbreviated as follows: 

CACM = Communications of the Association for Computing Machinery 
JACM = Journal of the Association for Computing Machinery 
Comp. J. — The Computer Journal (British Computer Society) 

Math. Comp. — Mathematics of Computation 

AMM — American Mathematical Monthly 

SICOMP = SIAM Journal on Computing 

FOCS — IEEE Symposium on Foundations of Computer Science 

SODA = ACM-SIAM Symposium on Discrete Algorithms 

STOC = ACM Symposium on Theory of Computing 

Crelle = Journal fur die reine und angewandte Mathematik 

As an example, “CACM 6 (1963), 555-563” stands for the reference given in a 
preceding paragraph of this preface. I also use ll CMath” to stand for the book 
Concrete Mathematics, which is cited in the introduction to Section 1.2. 

Much of the technical content of these books appears in the exercises. When 
the idea behind a nontrivial exercise is not my own, I have attempted to give 
credit to the person who originated that idea. Corresponding references to the 
literature are usually given in the accompanying text of that section, or in the 
answer to that exercise, but in many cases the exercises are based on unpublished 
material for which no further reference can be given. 

I have, of course, received assistance from a great many people during the 
years I have been preparing these books, and for this I am extremely thankful. 
Acknowledgments are due, first, to my wife, Jill, for her infinite patience, for 
preparing several of the illustrations, and for untold further assistance of all 
kinds; secondly, to Robert W. Floyd, who contributed a great deal of his time 
towards the enhancement of this material during the 1960s. Thousands of other 
people have also provided significant help — it would take another book just 
to list their names! Many of them have kindly allowed me to make use of 
hitherto unpublished work. My research at Caltech and Stanford was gener- 
ously supported for many years by the National Science Foundation and the 
Office of Naval Research. Addison- Wesley has provided excellent assistance and 
cooperation ever since I began this project in 1962. The best way I know how 
to thank everyone is to demonstrate by this publication that their input has led 
to books that resemble what I think they wanted me to write. 
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Preface to the Third Edition 

After having spent ten years developing the TgX and METRFONT systems for 
computer typesetting, I am now able to fulfill the dream that I had when I began 
that work, by applying those systems to The Art of Computer Programming. 
At last the entire text of this book has been captured inside my personal com- 
puter, in an electronic form that will make it readily adaptable to future changes 
in printing and display technology. The new setup has allowed me to make 
literally thousands of improvements that I have been wanting to incorporate for 
a long time. 

In this new edition I have gone over every word of the text, trying to retain 
the youthful exuberance of my original sentences while perhaps adding some 
more mature judgment. Dozens of new exercises have been added; dozens of old 
exercises have been given new and improved answers. 

f T he Art of Computer Programming is, however, still a work in progress. 

Therefore some parts of this book are headed by an “under construction” 
icon, to apologize for the fact that the material is not up-to-date. My files are 
bursting with important material that I plan to include in the final, glorious, 
fourth edition of Volume 1, perhaps 15 years from now; but I must finish 
Volumes 4 and 5 first, and I do not want to delay their publication any more 
than absolutely necessary. 

Most of the hard work of preparing the new edition was accomplished by 
Phyllis Winkler and Silvio Levy, who expertly keyboarded and edited the text 
of the second edition, and by Jeffrey Oldham, who converted nearly all of the 
original illustrations to METRPOST format. I have corrected every error that 
alert readers detected in the second edition (as well as some mistakes that, alas, 
nobody noticed); and I have tried to avoid introducing new errors in the new 
material. However, I suppose some defects still remain, and I want to fix them 
as soon as possible. Therefore I will cheerfully pay $2.56 to the first finder of 
each technical, typographical, or historical error. The webpage cited on page iv 
contains a current listing of all corrections that have been reported to me. 

Stanford, California D. E. K. 

April 1997 


Things have changed in the past two decades. 

— BILL GATES (1995) 



Flow chart for reading this set of books. 
















Procedure for Reading 
This Set of Books 


1. Begin reading this procedure, unless you have already begun to read it. 
Continue to follow the steps faithfully. (The general form of this procedure 
and its accompanying flow chart will be used throughout this book.) 

2 . Read the Notes on the Exercises, on pages xv-xvii. 

3. Set N equal to 1. 

4. Begin reading Chapter N. Do not read the quotations that appear at the 
beginning of the chapter. 

5. Is the subject of the chapter interesting to you? If so, go to step 7; if not, 
go to step 6. 

6. Is IV < 2? If not, go to step 16; if so, scan through the chapter anyway. 
(Chapters 1 and 2 contain important introductory material and also a review 
of basic programming techniques. You should at least skim over the sections 
on notation and about MIX.) 

7 . Begin reading the next section of the chapter; if you have already reached 
the end of the chapter, however, go to step 16. 

8. Is section number marked with “*”? If so, you may omit this section on 
first reading (it covers a rather specialized topic that is interesting but not 
essential); go back to step 7. 

9. Are you mathematically inclined? If math is all Greek to you, go to step 11; 
otherwise proceed to step 10. 

10 . Check the mathematical derivations made in this section (and report errors 
to the author). Go to step 12. 

11 . If the current section is full of mathematical computations, you had better 
omit reading the derivations. However, you should become familiar with the 
basic results of the section; they are usually stated near the beginning, or 
in slanted type right at the very end of the hard parts. 

12 . Work the recommended exercises in this section in accordance with the hints 
given in the Notes on the Exercises (which you read in step 2). 

13 . After you have worked on the exercises to your satisfaction, check your 
answers with the answer printed in the corresponding answer section at the 
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rear of the book (if any answer appears for that problem). Also read the 
answers to the exercises you did not have time to work. Note: In most cases 
it is reasonable to read the answer to exercise n before working on exercise 
n + 1, so steps 12-13 are usually done simultaneously. 

14 . Are you tired? If not, go back to step 7. 

15 . Go to sleep. Then, wake up, and go back to step 7. 

16 . Increase N by one. If TV = 3, 5, 7, 9, 11, or 12, begin the next volume of 
this set of books. 

17 . If TV is less than or equal to 12, go back to step 4. 

18 . Congratulations. Now try to get your friends to purchase a copy of Volume 1 
and to start reading it. Also, go back to step 3. 


Woe be to him that reads but one book. 
— GEORGE HERBERT, Jacula Prudentum, 1144 (1640) 

Le dgfaut unique de tous les ouvrages 
c’est d'etre trop longs. 
— VAUVENARGUES, Reflexions, 628 (1746) 

Books are a triviality. Life alone is great. 
— THOMAS CARLYLE, Journal (1839) 



NOTES ON THE EXERCISES 


The EXERCISES in this set of books have been designed for self-study as well as 
classroom study. It is difficult, if not impossible, for anyone to learn a subject 
purely by reading about it, without applying the information to specific problems 
and thereby being encouraged to think about what has been read. Furthermore, 
we all learn best the things that we have discovered for ourselves. Therefore the 
exercises form a major part of this work; a definite attempt has been made to 
keep them as informative as possible and to select problems that are enjoyable 
as well as instructive. 

In many books, easy exercises are found mixed randomly among extremely 
difficult ones. This is sometimes unfortunate because readers like to know in 
advance how long a problem ought to take — otherwise they may just skip over 
all the problems. A classic example of such a situation is the book Dynamic 
Programming by Richard Bellman; this is an important, pioneering work in 
which a group of problems is collected together, at the end of some chapters 
under the heading “Exercises and Research Problems,” with extremely trivial 
questions appearing in the midst of deep, unsolved problems. It is rumored that 
someone once asked Dr. Bellman how to tell the exercises apart from the research 
problems, and he replied, “If you can solve it, it is an exercise; otherwise it’s a 
research problem.” 

Good arguments can be made for including both research problems and 
very easy exercises in a book of this kind; therefore, to save the reader from 
the possible dilemma of determining which are which, rating numbers have been 
provided to indicate the level of difficulty. These numbers have the following 
general significance: 

Rating Interpretation 

00 An extremely easy exercise that can be answered immediately if the 
material of the text has been understood; such an exercise can almost 
always be worked “in your head.” 

10 A simple problem that makes you think over the material just read, but 
is by no means difficult. You should be able to do this in one minute at 
most; pencil and paper may be useful in obtaining the solution. 

20 An average problem that tests basic understanding of the text mate- 
rial, but you may need about fifteen or twenty minutes to answer it 
completely. 


XV 


XVI 


NOTES ON THE EXERCISES 


SO A problem of moderate difficulty and/or complexity; this one may 
involve more than two hours’ work to solve satisfactorily, or even more 
if the TV is on. 

40 Quite a difficult or lengthy problem that would be suitable for a term 
project in classroom situations. A student should be able to solve the 
problem in a reasonable amount of time, but the solution is not trivial. 

50 A research problem that has not yet been solved satisfactorily, as far 
as the author knew at the time of writing, although many people have 
tried. If you have found an answer to such a problem, you ought to 
write it up for publication; furthermore, the author of this book would 
appreciate hearing about the solution as soon as possible (provided that 
it is correct). 

By interpolation in this “logarithmic” scale, the significance of other rating 
numbers becomes clear. For example, a rating of if 7 would indicate an exercise 
that is a bit simpler than average. Problems with a rating of 50 that are 
subsequently solved by some reader may appear with a 45 rating in later editions 
of the book, and in the errata posted on the Internet (see page iv). 

The remainder of the rating number divided by 5 indicates the amount of 
detailed work required. Thus, an exercise rated 24 may take longer to solve than 
an exercise that is rated 25, but the latter will require more creativity. 

The author has tried earnestly to assign accurate rating numbers, but it is 
difficult for the person who makes up a problem to know just how formidable it 
will be for someone else to find a solution; and everyone has more aptitude for 
certain types of problems than for others. It is hoped that the rating n um bers 
represent a good guess at the level of difficulty, but they should be taken as 
general guidelines, not as absolute indicators. 

This book has been written for readers with varying degrees of mathematical 
training and sophistication; as a result, some of the exercises are intended only for 
the use of more mathematically inclined readers. The rating is preceded by an M 
if the exercise involves mathematical concepts or motivation to a greater extent 
than necessary for someone who is primarily interested only in programming 
the algorithms themselves. An exercise is marked with the letters “ HM ” if its 
solution necessarily involves a knowledge of calculus or other higher mathematics 
not developed in this book. An “/ZM” designation does not necessarily imply 
difficulty. 

Some exercises are preceded by an arrowhead, this designates prob- 
lems that are especially instructive and especially recommended. Of course, no 
reader/student is expected to work all of the exercises, so those that seem to be 
the most valuable have been singled out. (This is not meant to detract from the 
other exercises!) Each reader should at least make an attempt to solve all of the 
problems whose rating is Id or less; and the arrows may help to indicate which 
of the problems with a higher rating should be given priority. 

Solutions to most of the exercises appear in the answer section. Please use 
them wisely; do not turn to the answer until you have made a genuine effort to 
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solve the problem by yourself, or unless you absolutely do not have time to work 
this particular problem. After getting your own solution or giving the problem a 
decent try, you may find the answer instructive and helpful. The solution given 
will often be quite short, and it will sketch the details under the assumption 
that you have earnestly tried to solve it by your own means first. Sometimes the 
solution gives less information than was asked; often it gives more. It is quite 
possible that you may have a better answer than the one published here, or you 
may have found an error in the published solution; in such a case, the author 
will be pleased to know the details. Later editions of this book will give the 
improved solutions together with the solver’s name where appropriate. 

When working an exercise you may generally use the answers to previous 
exercises, unless specifically forbidden from doing so. The rating numbers have 
been assigned with this in mind; thus it is possible for exercise n + 1 to have a 
lower rating than exercise n, even though it includes the result of exercise n as 
a special case. 


Summary of codes: 

00 

Immediate 



10 

Simple (one minute) 



20 

Medium (quarter hour) 

► 

Recommended 

30 

Moderately hard 

M 

Mathematically oriented 

40 

Term project 

HM 

Requiring “higher math” 

50 

Research problem 


EXERCISES 

► 1. [00] What does the rating “ M20 ” mean? 

2. [10] Of what value can the exercises in a textbook be to the reader? 

3. [14] Prove that 13 3 = 2197. Generalize your answer. [This is an example of a 
horrible kind of problem that the author has tried to avoid.] 

4. [HM45] Prove that when n is an integer, n > 2, the equation x n + y n = z n has 
no solution in positive integers x, y, z. 


We can face our problem. 
We can arrange such facts as we have 
with order and method. 

— HERCULE POIROT, in Murder on the Orient Express (1934) 
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CHAPTER ONE 


BASIC CONCEPTS 


Many persons who are not conversant with mathematical studies 
imagine that because the business of [Babbage's Analytical Engine] is to 
give its results in numerical notation, the nature of its processes must 
consequently be arithmetical and numerical, rather than algebraical and 
analytical. This is an error. The engine can arrange and combine its 
numerical quantities exactly as if they were letters or any other general 
symbols; and in fact it might bring out its results in algebraical notation, 

were provisions made accordingly. 

— AUGUSTA ADA, Countess of Lovelace (1843) 

Practice yourself, for heaven's sake, in little things; 

and thence proceed to greater. 

— EPICTETUS ( Discourses IV.i) 


1.1. ALGORITHMS 

The notion of an algorithm is basic to all of computer programming, so we 
should begin with a careful analysis of this concept. 

The word “algorithm” itself is quite interesting; at first glance it may look 
as though someone intended to write “logarithm” but jumbled up the first four 
letters. The word did not appear in Webster’s New World Dictionary as late as 
1957; we find only the older form “algorism” with its ancient meaning, the process 
of doing arithmetic using Arabic numerals. During the Middle Ages, abacists 
computed on the abacus and algorists computed by algorism. By the time of the 
Renaissance, the origin of this word was in doubt, and early linguists attempted 
to guess at its derivation by making combinations like algiros [painful] + arithmos 
[number]; others said no, the word comes from “King Algor of Castile.” Finally, 
historians of mathematics found the true origin of the word algorism: It comes 
from the name of a famous Persian textbook author, Abu ‘Abd Allah Muhammad 
ibn Musa al-Khwarizml (c. 825) — literally, “Father of Abdullah, Mohammed, 
son of Moses, native of Khwarizm.” The Aral Sea in Central Asia was once 
known as Lake Khwarizm, and the Khwarizm region is located in the Amu River 
basin just south of that sea. Al-Khwarizml wrote the celebrated Arabic text 
Kitab al-jabr wa’l-muqabala (“Rules of restoring and equating”); another word, 
“algebra,” stems from the title of that book, which was a systematic study of the 
solution of linear and quadratic equations. [For notes on al-Khwarizml’s life and 
work, see H. Zemanek, Lecture Notes in Computer Science 122 (1981), 1-81.] 
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Gradually the form and meaning of algorism became corrupted; as ex- 
plained by the Oxford English Dictionary, the word “passed through many 
pseudo-etymological perversions, including a recent algorithm, in which it is 
learnedly confused” with the Greek root of the word arithmetic. This change 
from “algorism” to “algorithm” is not hard to understand in view of the fact 
that people had forgotten the original derivation of the word. An early German 
mathematical dictionary, Vollstandiges mathematisch.es Lexicon (Leipzig: 1747), 
gave the following definition for the word Algorithmus : “Under this designation 
are combined the notions of the four types of arithmetic calculations, namely 
addition, multiplication, subtraction, and division.” The Latin phrase algorith- 
mus infinitesimalis was at that time used to denote “ways of calculation with 
infinitely small quantities, as invented by Leibniz.” 

By 1950, the word algorithm was most frequently associated with Euclid’s 
algorithm, a process for finding the greatest common divisor of two numbers 
that appears in Euclid’s Elements (Book 7, Propositions 1 and 2). It will be 
instructive to exhibit Euclid’s algorithm here: 

Algorithm E ( Euclid’s algorithm). Given two positive integers m and n, find 
their greatest common divisor, that is, the largest positive integer that evenly 
divides both m and n. 

El. [Find remainder.] Divide m by n and let r be the remainder. (We will have 

0 < r < n.) 

E2. [Is it zero?] If r = 0, the algorithm terminates; n is the answer. 

E3. [Reduce.] Set m <— n, n «— r, and go back to step El. | 

Of course, Euclid did not present his algorithm in just this manner. The 
format above illustrates the style in which all of the algorithms throughout this 
book will be presented. 

Each algorithm we consider has been given an identifying letter (E in the 
preceding example), and the steps of the algorithm are identified by this letter 
followed by a number (El, E2, E3). The chapters are divided into numbered 
sections; within a section the algorithms are designated by letter only, but when 
algorithms are referred to in other sections, the appropriate section number is 
attached. For example, we are now in Section 1.1; within this section Euclid’s 
algorithm is called Algorithm E, while in later sections it is referred to as 
Algorithm 1.1E. 

Each step of an algorithm, such as step El above, begins with a phrase in 
brackets that sums up as briefly as possible the principal content of that step. 
This phrase also usually appears in an accompanying flow chart, such as Fig. 1, 
so that the reader will be able to picture the algorithm more readily. 

After the summarizing phrase comes a description in words and symbols 
of some action to be performed or some decision to be made. Parenthesized 
comments, like the second sentence in step El, may also appear. Comments are 
included as explanatory information about that step, often indicating certain 
invariant characteristics of the variables or the current goals at that step. They 


1.1 


ALGORITHMS 


3 



Fig. 1. Flow chart for Algorithm E. 

do not specify actions that belong to the algorithm, but are meant only for the 
reader’s benefit as possible aids to comprehension. 

The arrow “ 4— ” in step E3 is the all-important replacement operation, some- 
times called assignment or substitution: “m 4— n” means that the value of 
variable m is to be replaced by the current value of variable n. When Algorithm E 
begins, the values of m and n are the originally given numbers; but when it 
ends, those variables will have, in general, different values. An arrow is used 
to distinguish the replacement operation from the equality relation: We will 
not say, “Set m — n,” but we will perhaps ask, “Does m — n?” The “ = ” 
sign denotes a condition that can be tested, the “ 4— ” sign denotes an action 
that can be performed. The operation of increasing n by one is denoted by 
“n 4— n + 1” (read “n is replaced by n + 1” or “n gets n + 1”). In general, 
“variable 4— formula” means that the formula is to be computed using the present 
values of any variables appearing within it; then the result should replace the 
previous value of the variable at the left of the arrow. Persons untrained in 
computer work sometimes have a tendency to say “n becomes n + 1” and to 
write “n-)ni 1” for the operation of increasing n by one; this symbolism can 
only lead to confusion because of its conflict with standard conventions, and it 
should be avoided. 

Notice that the order of actions in step E3 is important: “Set m 4— n, 
n 4— r” is quite different from “Set n 4— r, m 4— n,” since the latter would imply 
that the previous value of n is lost before it can be used to set to. Thus the 
latter sequence is equivalent to “Set n 4— r, m 4— r.” When several variables 
are all to be set equal to the same quantity, we can use multiple arrows; for 
example, “n 4— r, m 4— r” may be written “n 4— to 4- r.” To interchange the 
values of two variables, we can write “Exchange m 4-> n” ; this action could also 
be specified by using a new variable t and writing “Set 1 4— to, to 4— n, n 4— t." 

An algorithm starts at the lowest- numbered step, usually step 1, and it 
performs subsequent steps in sequential order unless otherwise specified. In step 
E3, the imperative “go back to step El” specifies the computational order in an 
obvious fashion. In step E2, the action is prefaced by the condition “If r = 0”; 
so if r / 0, the rest of that sentence does not apply and no action is specified. 
We might have added the redundant sentence, “If r / 0, go on to step E3.” 

The heavy vertical line “ | ” appearing at the end of step E3 is used to 
indicate the end of an algorithm and the resumption of text. 

We have now discussed virtually all the notational conventions used in the 
algorithms of this book, except for a notation used to denote “subscripted” or 
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“indexed” items that are elements of an ordered array. Suppose we have n 
quantities, v\, v 2 , . . . , v n ] instead of writing Vj for the j th element, the notation 
v[j] is often used. Similarly, a[i,j] is sometimes used in preference to a doubly 
subscripted notation like a^. Sometimes multiple- letter names are used for 
variables, usually set in capital letters; thus TEMP might be the name of a variable 
used for temporarily holding a computed value, PRIME [K] might denote the Kth 
prime number, and so on. 

So much for the form of algorithms; now let us perform one. It should be 
mentioned immediately that the reader should not expect to read an algorithm 
as if it were part of a novel; such an attempt would make it pretty difficult to 
understand what is going on. An algorithm must be seen to be believed, and the 
best way to learn what an algorithm is all about is to try it. The reader should 
always take pencil and paper and work through an example of each algorithm 
immediately upon encountering it in the text. Usually the outline of a worked 
example will be given, or else the reader can easily conjure one up. This is a 
simple and painless way to gain an understanding of a given algorithm, and all 
other approaches are generally unsuccessful. 

Let us therefore work out an example of Algorithm E. Suppose that we are 
given m = 119 and n = 544; we are ready to begin, at step El. (The reader 
should now follow the algorithm as we give a play-by-play account.) Dividing 
m by n in this case is quite simple, almost too simple, since the quotient is zero 
and the remainder is 119. Thus, r 4 - 119. We proceed to step E2, and since 
r ^ 0 no action occurs. In step E3 we set m 4 - 544, n 4- 119. It is clear that if 
m < n originally, the quotient in step El will always be zero and the algorithm 
will always proceed to interchange m and n in this rather cumbersome fashion. 
We could add a new step: 

EO. [Ensure m > n.] If m < n, exchange m 4-t n. 

This would make no essential change in the algorithm, except to increase its 
length slightly, and to decrease its running time in about one half of all cases. 

Back at step El, we find that 544/119 = 4 + 68/119, so r 4- 68. Again E2 is 
inapplicable, and at E3 we set m 4— 119, n 4— 68. The next round sets r 4 — 51, 
and ultimately m 4 - 68, n 4 - 51. Next r 4 - 17, and m 4 - 51, n 4- 17. Finally, 
when 51 is divided by 17, we set r 4 — 0, so at step E2 the algorithm terminates. 
The greatest common divisor of 119 and 544 is 17. 

So this is an algorithm. The modern meaning for algorithm is quite similar to 
that of recipe, process, method, technique, procedure, routine, rigmarole, except 
that the word “algorithm” connotes something just a little different. Besides 
merely being a finite set of rules that gives a sequence of operations for solving 
a specific type of problem, an algorithm has five important features: 

1) Finiteness. An algorithm must always terminate after a finite number of 
steps. Algorithm E satisfies this condition, because after step El the value of r 
is less than n; so if r / 0, the value of n decreases the next time step El is 
encountered. A decreasing sequence of positive integers must eventually termi- 
nate, so step El is executed only a finite number of times for any given original 
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value of n. Note, however, that the number of steps can become arbitrarily large; 
certain huge choices of to and n will cause step El to be executed more than a 
million times. 

(A procedure that has all of the characteristics of an algorithm except that it 
possibly lacks finiteness may be called a computational method. Euclid originally 
presented not only an algorithm for the greatest common divisor of numbers, but 
also a very similar geometrical construction for the “greatest common measure” 
of the lengths of two line segments; this is a computational method that does 
not terminate if the given lengths are incommensurable. Another example of a 
nonterminating computational method is a reactive process , which continually 
interacts with its environment.) 

2) Definiteness. Each step of an algorithm must be precisely defined; the ac- 
tions to be carried out must be rigorously and unambiguously specified for each 
case. The algorithms of this book will hopefully meet this criterion, but they 
are specified in the English language, so there is a possibility that the reader 
might not understand exactly what the author intended. To get around this 
difficulty, formally defined programming languages or computer languages are 
designed for specifying algorithms, in which every statement has a very definite 
meaning. Many of the algorithms of this book will be given both in English 
and in a computer language. An expression of a computational method in a 
computer language is called a program. 

In Algorithm E, the criterion of definiteness as applied to step El means that 
the reader is supposed to understand exactly what it means to divide m by n 
and what the remainder is. In actual fact, there is no universal agreement about 
what this means if to and n are not positive integers; what is the remainder of 
—8 divided by —it? What is the remainder of 59/13 divided by zero? Therefore 
the criterion of definiteness means we must make sure that the values of m and n 
are always positive integers whenever step El is to be executed. This is initially 
true, by hypothesis; and after step El, r is a nonnegative integer that must be 
nonzero if we get to step E3. So m and n are indeed positive integers as required. 

3) Input. An algorithm has zero or more inputs : quantities that are given to it 
initially before the algorithm begins, or dynamically as the algorithm runs. These 
inputs are taken from specified sets of objects. In Algorithm E, for example, there 
are two inputs, namely to and n, both taken from the set of positive integers. 

4) Output. An algorithm has one or more outputs: quantities that have a 
specified relation to the inputs. Algorithm E has one output, namely n in step E2, 
the greatest common divisor of the two inputs. 

(We can easily prove that this number is indeed the greatest common divisor, 
as follows. After step El, we have 


m — qn + r, 

for some integer q. If r = 0, then to is a multiple of n, and clearly in such a case 
n is the greatest common divisor of to and n. If r 0, note that any number 
that divides both to and n must divide to — qn = r, and any number that divides 
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both n and r must divide qn + r = m; so the set of common divisors of {m, n} 
is the same as the set of common divisors of {n,r}. In particular, the greatest 
common divisor of {m,n} is the same as the greatest common divisor of {n,r}. 
Therefore step E3 does not change the answer to the original problem.) 

5 ) Effectiveness. An algorithm is also generally expected to be effective, in the 
sense that its operations must all be sufficiently basic that they can in principle 
be done exactly and in a finite length of time by someone using pencil and 
paper. Algorithm E uses only the operations of dividing one positive integer 
by another, testing if an integer is zero, and setting the value of one variable 
equal to the value of another. These operations are effective, because integers 
can be represented on paper in a finite manner, and because there is at least 
one method (the “division algorithm”) for dividing one by another. But the 
same operations would not be effective if the values involved were arbitrary real 
numbers specified by an infinite decimal expansion, nor if the values were the 
lengths of physical line segments (which cannot be specified exactly). Another 
example of a noneffective step is, “If 4 is the largest integer n for which there is 
a solution to the equation w n +x n + y n = z n in positive integers w, x, y, and z, 
then go to step E4.” Such a statement would not be an effective operation until 
someone successfully constructs an algorithm to determine whether 4 is or is not 
the largest integer with the stated property. 

Let us try to compare the concept of an algorithm with that of a cookbook 
recipe. A recipe presumably has the qualities of finiteness (although it is said 
that a watched pot never boils), input (eggs, flour, etc.), and output (TV dinner, 
etc.), but it notoriously lacks definiteness. There are frequent cases in which a 
cook’s instructions are indefinite: “Add a dash of salt.” A “dash” is defined 
to be “less than 1 /s teaspoon,” and salt is perhaps well enough defined; but 
where should the salt be added — on top? on the side? Instructions like “toss 
lightly until mixture is crumbly” or “warm cognac in small saucepan” are quite 
adequate as explanations to a trained chef, but an algorithm must be specified 
to such a degree that even a computer can follow the directions. Nevertheless, 
a computer programmer can learn much by studying a good recipe book. (The 
author has in fact barely resisted the temptation to name the present volume 
“The Programmer’s Cookbook.” Perhaps someday he will attempt a book called 
“Algorithms for the Kitchen.”) 

We should remark that the finiteness restriction is not really strong enough 
for practical use. A useful algorithm should require not only a finite number 
of steps, but a very finite number, a reasonable number. For example, there is 
an algorithm that determines whether or not the game of chess can always be 
won by White if no mistakes are made (see exercise 2.2.3-28). That algorithm 
can solve a problem of intense interest to thousands of people, yet it is a safe 
bet that we will never in our lifetimes know the answer; the algorithm requires 
fantastically large amounts of time for its execution, even though it is finite. See 
also Chapter 8 for a discussion of some finite numbers that are so large as to 
actually be beyond comprehension. 
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In practice we not only want algorithms, we want algorithms that are good 
in some loosely defined aesthetic sense. One criterion of goodness is the length 
of time taken to perform the algorithm; this can be expressed in terms of the 
number of times each step is executed. Other criteria are the adaptability of the 
algorithm to different kinds of computers, its simplicity and elegance, etc. 

We often are faced with several algorithms for the same problem, and we 
must decide which is best. This leads us to the extremely interesting and 
all-important field of algorithmic analysis: Given an algorithm, we want to 
determine its performance characteristics. 

For example, let’s consider Euclid’s algorithm from this point of view. Sup- 
pose we ask the question, “Assuming that the value of n is known but m is 
allowed to range over all positive integers, what is the average number of times, 
T„, that step El of Algorithm E will be performed?” In the first place, we need 
to check that this question does have a meaningful answer, since we are trying 
to take an average over infinitely many choices for to. But it is evident that 
after the first execution of step El only the remainder of to after division by n is 
relevant. So all we must do to find T„ is to try the algorithm for rri — 1, to = 2, 
. . . , to = n, count the total number of times step El has been executed, and 
divide by n. 

Now the important question is to determine the nature of T„; is it approxi- 
mately equal to |n, or sjn, for instance? As a matter of fact, the answer to this 
question is an extremely difficult and fascinating mathematical problem, not yet 
completely resolved, which is examined in more detail in Section 4.5.3. For large 
values of n it is possible to prove that T n is approximately (l2(ln2)/7r 2 ) Inn, 
that is, proportional to the natural logarithm of n, with a constant of propor- 
tionality that might not have been guessed offhand! For further details about 
Euclid’s algorithm, and other ways to calculate the greatest common divisor, see 
Section 4.5.2. 

Analysis of algorithms is the name the author likes to use to describe 
investigations such as this. The general idea is to take a particular algorithm 
and to determine its quantitative behavior; occasionally we also study whether 
or not an algorithm is optimal in some sense. The theory of algorithms is another 
subject entirely, dealing primarily with the existence or nonexistence of effective 
algorithms to compute particular quantities. 

So far our discussion of algorithms has been rather imprecise, and a mathe- 
matically oriented reader is justified in thinking that the preceding commentary 
makes a very shaky foundation on which to erect any theory about algorithms. 
We therefore close this section with a brief indication of one method by which the 
concept of algorithm can be firmly grounded in terms of mathematical set theory. 
Let us formally define a computational method to be a quadruple {Q,I, SI, /), 
in which Q is a set containing subsets I and 0, and / is a function from Q 
into itself. Furthermore / should leave pointwise fixed; that is, /(g) should 
equal g for all elements g of SI. The four quantities Q, /, Q, / are intended 
to represent respectively the states of the computation, the input, the output, 
and the computational rule. Each input x in the set I defines a computational 
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sequence, xo, %i, X 2 , ■ ■ ■ , as follows: 

Xq — x and Xk + 1 = f{xk) for k > 0 . (l) 

The computational sequence is said to terminate in k steps if k is the smallest 
integer for which Xk is in fl, and in this case it is said to produce the output 
Xk from x. (Note that if Xk is in f2, so is Xk+ 1 , because Xk+i — Xk in such a 
case.) Some computational sequences may never terminate; an algorithm is a 
computational method that terminates in finitely many steps for all x in I. 

Algorithm E may, for example, be formalized in these terms as follows: Let Q 
be the set of all singletons (n), all ordered pairs (m, n), and all ordered quadruples 
(m, n,r, 1), (m,n,r, 2), and (m,n,p, 3), where m, n, and p are positive integers 
and r is a nonnegative integer. Let I be the subset of all pairs (to, n) and let ft 
be the subset of all singletons (n). Let / be defined as follows: 

/((m,n)) = (m, n, 0,1); /((n)) = (n); 

/((m,n, r, 1 )) = (to, n, remainder of to divided by n, 2 ); 

(2) 

/((m, n, r, 2)) = (n) if r = 0, (m,n,r, 3) otherwise; 

/ ((to, n,p, 3)) = (n,p,p, 1). 

The correspondence between this notation and Algorithm E is evident. 

This formulation of the concept of an algorithm does not include the re- 
striction of effectiveness mentioned earlier. For example, Q might denote infinite 
sequences that are not computable by pencil and paper methods, or f might 
involve operations that mere mortals cannot always perform. If we wish to 
restrict the notion of algorithm so that only elementary operations are involved, 
we can place restrictions on Q, I, 0, and /, for example as follows: Let A be 
a finite set of letters, and let A* be the set of all strings on A (the set of all 
ordered sequences aqa^ • ■ . x n , where n > 0 and Xj is in A for 1 < j < n). The 
idea is to encode the states of the computation so that they are represented by 
strings of A*. Now let A’ be a nonnegative integer and let Q be the set of all 
(a,j), where a is in A* and j is an integer, 0 < j < N: let I be the subset of Q 
with j = 0 and let ff be the subset with j — N. If 6 and a are strings in A*, we 
say that 0 occurs in a if a has the form aOu for strings a and us. To complete 
our definition, let / be a function of the following type, defined by the strings 
Oj, <j)j and the integers aj, bj for 0 < j < N: 

/((<bi)) = ( a,aj ) if Oj does not occur in a; 

f((a,j)) = (a<t>jus,bj) if a is the shortest possible string for which a = aOjU>-, 
f((a,N)) = (a,N). ( 3 ) 

Such a computational method is clearly effective, and experience shows 
that it is also powerful enough to do anything we can do by hand. There are 
many other essentially equivalent ways to formulate the concept of an effective 
computational method (for example, using Turing machines). The formulation 
above is virtually the same as that given by A. A. Markov in his book The 
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Theory of Algorithms [Trudy Mat. Inst. Akad. Nauk 42 (1954), 1-376], later 
revised and enlarged by N. M. Nagorny (Moscow: Nauka, 1984; English edition, 
Dordrecht: Kluwer, 1988). 

EXERCISES 

1. [10] The text showed how to interchange the values of variables m and n, using 
the replacement notation, by setting t 4- m, m 4— n, n 4— t. Show how the values of 
four variables (a, 6, c, d) can be rearranged to (b. c, d. a) by a sequence of replacements. 
In other words, the new value of a is to be the original value of b, etc. Try to use the 
minimum number of replacements. 

2. [15] Prove that m is always greater than n at the beginning of step El, except 
possibly the first time this step occurs. 

3. [20] Change Algorithm E (for the sake of efficiency) so that all trivial replacement 
operations such as “m 4— n” are avoided. Write this new algorithm in the style of 
Algorithm E, and call it Algorithm F. 

4. [16] What is the greatest common divisor of 2166 and 6099? 

► 5. [12] Show that the “Procedure for Reading This Set of Books” that appears in 
the preface actually fails to be a genuine algorithm on three of our five counts! Also 
mention some differences in format between it and Algorithm E. 

6. [20] What is T5, the average number of times step El is performed when n = 5? 

► 7. [M21] Suppose that m is known and n is allowed to range over all positive integers; 
let Um be the average number of times that step El is executed in Algorithm E. Show 
that Um is well defined. Is U m in any way related to T m ? 

8. [M25] Give an “effective” formal algorithm for computing the greatest common 
divisor of positive integers m and n, by specifying 6j, <j>j , dj, bj as in Eqs. (3). Let the 
input be represented by the string a m b n , that is, m a’s followed by n b’s. Try to make 
your solution as simple as possible. [Hint: Use Algorithm E, but instead of division in 
step El, set r 4— |m — n|, n 4— min (m, n).] 

► 9. [ M30 ] Suppose that Ci = (Qi,/i,£2i,/i) and C2 = (Q2, h, D2, }i) are computa- 
tional methods. For example, Ci might stand for Algorithm E as in Eqs. (2), except 
that m and n are restricted in magnitude, and C 2 might stand for a computer program 
implementation of Algorithm E. (Thus Q 2 might be the set of all states of the machine, 
i.e., all possible configurations of its memory and registers; f 2 might be the definition 
of single machine actions; and I 2 might be the initial state, including the program for 
determining the greatest common divisor as well as the values of m and n.) 

Formulate a set-theoretic definition for the concept “C2 is a representation of Ci” 
or “C2 simulates Ci .” This is to mean intuitively that any computation sequence of C\ 
is mimicked by C 2 , except that C 2 might take more steps in which to do the computation 
and it might retain more information in its states. (We thereby obtain a rigorous 
interpretation of the statement, “Program X is an implementation of Algorithm Yd) 
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1.2. MATHEMATICAL PRELIMINARIES 

In this SECTION we shall investigate the mathematical notations that occur 
throughout The Art of Computer Programming, and we’ll derive several basic 
formulas that will be used repeatedly. Even a reader not concerned with the 
more complex mathematical derivations should at least become familiar with 
the meanings of the various formulas, so as to be able to use the results of the 
derivations. 

Mathematical notation is used for two main purposes in this book: to 
describe portions of an algorithm, and to analyze the performance character- 
istics of an algorithm. The notation used in descriptions of algorithms is quite 
simple, as explained in the previous section. When analyzing the performance 
of algorithms, we need to use other more specialized notations. 

Most of the algorithms we will discuss are accompanied by mathematical 
calculations that determine the speed at which the algorithm may be expected 
to run. These calculations draw on nearly every branch of mathematics, and a 
separate book would be necessary to develop all of the mathematical concepts 
that are used in one place or another. However, the majority of the calculations 
can be carried out with a knowledge of college algebra, and the reader with a 
knowledge of elementary calculus will be able to understand nearly all of the 
mathematics that appears. Sometimes we will need to use deeper results of 
complex variable theory, group theory, number theory, probability theory, etc.; 
in such cases the topic will be explained in an elementary manner, if possible, or 
a reference to other sources of information will be given. 

The mathematical techniques involved in the analysis of algorithms usually 
have a distinctive flavor. For example, we will quite often find ourselves working 
with finite summations of rational numbers, or with the solutions to recurrence 
relations. Such topics are traditionally given only a light treatment in mathe- 
matics courses, and so the following subsections are designed not only to give a 
thorough drilling in the use of the notations to be defined but also to illustrate 
in depth the types of calculations and techniques that will be most useful to us. 

Important note: Although the following subsections provide a rather extensive 
training in the mathematical skills needed in connection with the study of com- 
puter algorithms, most readers will not see at first any very strong connections 
between this material and computer programming (except in Section 1.2.1). The 
reader may choose to read the following subsections carefully, with implicit faith 
in the author’s assertion that the topics treated here are indeed very relevant; but 
it is probably preferable, for motivation, to skim over this section lightly at first, 
and (after seeing numerous applications of the techniques in future chapters) 
return to it later for more intensive study. If too much time is spent studying 
this material when first reading the book, a person might never get on to the 
computer programming topics! However, each reader should at least become 
familiar with the general contents of these subsections, and should try to solve a 
few of the exercises, even on first reading. Section 1.2.10 should receive particular 
attention, since it is the point of departure for most of the theoretical material 
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developed later. Section 1.3, which follows 1.2, abruptly leaves the realm of 
“pure mathematics” and enters into “pure computer programming.” 

An expansion and more leisurely presentation of much of the following 
material can be found in the book Concrete Mathematics by Graham, Knuth, 
and Patashnik, second edition (Reading, Mass.: Addison- Wesley, 1994). That 
book will be called simply CMath when we need to refer to it later. 

1.2.1. Mathematical Induction 

Let P{n) be some statement about the integer n; for example, P(n) might be 
“n times (n + 3) is an even number,” or “if n > 10, then 2" > n 3 .’ Suppose we 
want to prove that P(n) is true for all positive integers n. An important way to 
do this is: 

a) Give a proof that -P(l) is true. 

b) Give a proof that “if all of P(l), P( 2), . . . , P(n) are true, then P(n + 1) is 
also true”; this proof should be valid for any positive integer n. 

As an example, consider the following series of equations, which many people 
have discovered independently since ancient times: 

1 = 1 2 , 
l + 3 = 2 2 , 

1 4- 3 + 5 = 3 2 , 

1 + 3 + 5 + 7 = 4 2 , 

l + 3 + 5 + 7 + 9 = 5 2 . (i) 

We can formulate the general property as follows: 

1 + 3+ b(2n-l)=n 2 . ( 2 ) 

Let us, for the moment, call this equation P(n ); we wish to prove that P(n) is 
true for all positive n. Following the procedure outlined above, we have: 

a) “P(l) is true, since 1 = l 2 .” 

b) “If all of P(l), . . . , P(n) are true, then, in particular, P(n) is true, so Eq. ( 2 ) 
holds; adding 2n + 1 to both sides we obtain 

1 + 3 + b (2n - 1) + (2n + 1) .= n 2 + 2n + 1 = (n + l) 2 , 

which proves that P(n + 1) is also true.” 

We can regard this method as an algorithmic proof procedure. In fact, the 
following algorithm produces a proof of P{n) for any positive integer n, assuming 
that steps (a) and (b) above have been worked out: 

Algorithm I ( Construct a proof). Given a positive integer n, this algorithm 
will output a proof that Pin) is true. 

II. [Prove P(l).] Set k +- 1, and, according to (a), output a proof of P(l). 
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12 . [k = n?] If k = n, terminate the algorithm; the required proof has been 
output. 

13 . [Prove P(k + 1).] According to (b), output a proof that “If all of P{ 1), . . . , 
P{k) are true, then P(k + 1) is true.” Also output “We have already proved 
.P(l), . . . , P(k); hence P(k + 1) is true.” 

14 . [Increase k] Increase A; by 1 and go to step 12. | 



Fig. 2. Algorithm I: Mathematical induction. 

Since this algorithm clearly presents a proof of P(n), for any given u . the 
proof technique consisting of steps (a) and (b) is logically valid. It is called proof 
by mathematical induction. 

The concept of mathematical induction should be distinguished from what 
is usually called inductive reasoning in science. A scientist takes specific observa- 
tions and creates, by “induction,” a general theory or hypothesis that accounts 
for these facts; for example, we might observe the five relations in (i), above, 
and formulate ( 2 ). In this sense, induction is no more than our best guess about 
the situation; mathematicians would call it an empirical result or a conjecture. 

Another example will be helpful. Let p(n) denote the number of partitions 
of n, that is, the number of different ways to write n as a sum of positive integers, 
disregarding order. Since 5 can be partitioned in exactly seven ways, 

l + l + l + l + l = 2 + l + l + l = 2 + 2 + l = 3+l + l = 3 + 2 = 4 + l = 5, 

we have p( 5) = 7. In fact, it is easy to establish the first few values, 

P( 1) = 1. P( 2) = 2, p( 3) = 3, p( 4) = 5, p(S) = 7. 

At this point we might tentatively formulate, by induction, the hypothesis that 
the sequence p( 2), p( 3), ... runs through the prime numbers. To test this 
hypothesis, we proceed to calculate pi (> ) and behold! p(6) — 11, confirming 
our conjecture. 

[Unfortunately, p( 7) turns out to be 15, spoiling everything, and we must 
try again. The numbers p(ri) are known to be quite complicated, although 
S. Ramanujan succeeded in guessing and proving many remarkable things about 
them. For further information, see G. H. Hardy, Ramanujan (London: Cam- 
bridge University Press, 1940), Chapters 6 and 8. See also Section 7.2. 1.4.] 

Mathematical induction is quite different from induction in the sense just 
explained. It is not just guesswork, but a conclusive proof of a statement; indeed, 
it is a proof of infinitely many statements, one for each n. It has been called 
“induction” only because one must first decide somehow what is to be proved, 
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before one can apply the technique of mathematical induction. Henceforth in 
this book we shall use the word induction only when we wish to imply proof by 
mathematical induction. 

There is a geometrical way to prove Eq. (2). 

Figure 3 shows, for n = 6, n 2 cells broken into 
groups of 1 + 3 + • ■ • + (2 n — 1) cells. However, in 
the final analysis, this picture can be regarded as a 
“proof” only if we show that the construction can 
be carried out for all n, and such a demonstration 
is essentially the same as a proof by induction. 

Our proof of Eq. (2) used only a special case 
of (b); we merely showed that the truth of P{n) 
implies the truth of P(n + 1). This is an important 
simple case that arises frequently, but our next 
example illustrates the power of the method a little 
more. We define the Fibonacci sequence Fq, F\, 

F 2i ... by the rule that Fq — 0, Fi = 1, and every further term is the sum 
of the preceding two. Thus the sequence begins 0, 1, 1, 2, 3, 5, 8, 13, . . . ; we 
will investigate it in detail in Section 1.2.8. We will now prove that if <p is the 
number (1 + a/ 5 )/2 we have 

Fn < r- 1 (3) 

for all positive integers n. Call this formula P(n). 

If n = 1, then Fi = 1 = <j>° = 0 n_1 , so step (a) has been done. For step (b) 
we notice first that P{ 2) is also true, since F 2 = 1 < 1-6 < <f> x = </ _1 . Now, if all 
of -P(l), P{ 2), . . . , P{n) are true and n > 1, we know in particular that P(n — 1) 
and P(n) are true; so F„_i < cj) n ~ 2 and F n < </> n_1 . Adding these inequalities, 
we get 

F n +i = K-i +F n < r~ 2 + 0” _1 = /“ 2 (1 + </>)• (4) 

The important property of the number <p, indeed the reason we chose this number 
for this problem in the first place, is that 

1 + <t> = 4> 2 . (5) 

Plugging (5) into (4) gives F n+ i < <j > n , which is P(n + 1). So step (b) has 
been done, and (3) has been proved by mathematical induction. Notice that we 
approached step (b) in two different ways here: We proved P(n+1) directly when 
n= 1, and we used an inductive method when n > 1. This was necessary, since 
when n = 1 our reference to P(n — 1) = -P(O) would not have been legitimate. 

Mathematical induction can also be used to prove things about algorithms. 
Consider the following generalization of Euclid’s algorithm. 

Algorithm E ( Extended Euclid’s algorithm). Given two positive integers m 
and n, we compute their greatest common divisor d, and we also compute two 
not-necessarily-positive integers a and b such that am + bn — d. 

El. [Initialize.] Set a 1 4- b 1, a <— b' «— 0, c m, d <— n. 
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Fig. 3. The sum of odd 
numbers is a square. 
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E2. [Divide.] Let q and r be the quotient and remainder, respectively, of c 
divided by d. (We have c — qd + r and 0 < r < d.) 

E3. [Remainder zero?] If r = 0, the algorithm terminates; we have in this case 
am + bn = d as desired. 

E4. [Recycle.] Set c <- d, d <- r, t <- a', a 1 <- a, a <- t - qa, t <- b', b' <- b, 
b t — qb, and go back to E2. | 

If we suppress the variables a, b, a', and b' from this algorithm and use m 
and n for the auxiliary variables c and d, we have our old algorithm, 1.1E. The 
new version does a little more, by determining the coefficients a and b. Suppose 
that m = 1769 and n = 551; we have successively (after step E2): 

a! a b 1 b c d q r 


1 

0 

0 

1 

1769 

551 

3 

116 

0 

1 

1 

-3 

551 

116 

4 

87 

1 

-4 

-3 

13 

116 

87 

1 

29 

4 

5 

13 

-16 

87 

29 

3 

0 


The answer is correct: 5 x 1769 - 16 x 551 = 8845 - 8816 = 29, the greatest 
common divisor of 1769 and 551. 

The problem is to prove that this algorithm works properly, for all m and n. 
We can try to apply the method of mathematical induction by letting P(n) be 
the statement “Algorithm E works for n and all integers m.” However, that 
approach doesn’t work out so easily, and we need to prove some extra facts. 
After a little study, we find that something must be proved about a, b, a', and 
b', and the appropriate fact is that the equalities 

a'm + b'n = c, am + bn = d (6) 

always hold whenever step E2 is executed. We may prove these equalities directly 
by observing that they are certainly true the first time we get to E2, and that 
step E4 does not change their validity. (See exercise 6.) 

Now we are ready to show that Algorithm E is valid, by induction on n: If 
m is a multiple of n, the algorithm obviously works properly, since we are done 
immediately at E3 the first time. This case always occurs when n = 1. The 
only case remaining is when n > 1 and m is not a multiple of n. In such a 
case, the algorithm proceeds to set c <- n, d <- r after the first execution, and 
since r < n, we may assume by induction that the final value of d is the gcd 
of n and r. By the argument given in Section 1.1, the pairs {m,n} and {n,r} 
have the same common divisors, and, in particular, they have the same greatest 
common divisor. Hence d is the gcd of m and n, and am + bn = d by (6). 

The italicized phrase in the proof above illustrates the conventional lan- 
guage that is so often used in an inductive proof: When doing part (b) of the 
construction, rather than saying “We will now assume P( 1), P(2), . . . , P(n), and 
with this assumption we will prove P(n + 1),” we often say simply “We will now 
prove P(n); we may assume by induction that P(k) is true whenever 1 < k < n.” 
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Fig. 4. Flow chart for Algorithm E, labeled with assertions that prove the validity of 
the algorithm. 

If we examine this argument very closely and change our viewpoint slightly, 
we can envision a general method applicable to proving the validity of any 
algorithm. The idea is to take a flow chart for some algorithm and to label 
each of the arrows with an assertion about the current state of affairs at the 
time the computation traverses that arrow. See Fig. 4, where the assertions 
have been labeled Al, A2, . . . , A6. (All of these assertions have the additional 
stipulation that the variables are integers; this stipulation has been omitted to 
save space.) A1 gives the initial assumptions upon entry to the algorithm, and 
A4 states what we hope to prove about the output values a, b, and d. 

The general method consists of proving, for each box in the flow chart, that 

if an assertion attached to any arrow leading into the box is true 
before the operation in that box is performed, then all of the , , 
assertions on relevant arrows leading away from the box are true ' 
after the operation. 

Thus, for example, we must prove that either A2 or A 6 before E2 implies AS 
after E2. (In this case A2 is a stronger statement than A6\ that is, A2 implies 
A6. So we need only prove that A6 before E2 implies AS after. Notice that the 
condition d > 0 is necessary in A 6 just to prove that operation E2 even makes 
sense.) It is also necessary to show that AS and r = 0 implies A 4 ; that AS and 
r 0 implies A5 ; etc. Each of the required proofs is very straightforward. 

Once statement (7) has been proved for each box, it follows that all assertions 
are true during any execution of the algorithm. For we can now use induction 





16 


BASIC CONCEPTS 


1 . 2.1 


on the number of steps of the computation, in the sense of the number of arrows 
traversed in the flow chart. While traversing the first arrow, the one leading from 
“Start”, the assertion Al is true since we always assume that our input values 
meet the specifications; so the assertion on the first arrow traversed is correct. 
If the assertion that labels the nth arrow is true, then by ( 7 ) the assertion that 
labels the (n + l)st arrow is also true. 

Using this general method, the problem of proving that a given algorithm 
is valid evidently consists mostly of inventing the right assertions to put in the 
flow chart. Once this inductive leap has been made, it is pretty much routine to 
carry out the proofs that each assertion leading into a box logically implies each 
assertion leading out. In fact, it is pretty much routine to invent the assertions 
themselves, once a few of the difficult ones have been discovered; thus it is very 
simple in our example to write out essentially what A2 , A3, and A 5 must be, 
if only Al, A 4 , and A 6 are given. In our example, assertion A 6 is the creative 
part of the proof; all the rest could, in principle, be supplied mechanically. Hence 
no attempt has been made to give detailed formal proofs of the algorithms that 
follow in this book, at the level of detail found in Fig. 4. It suffices to state 
the key inductive assertions. Those assertions either appear in the discussion 
following an algorithm or they are given as parenthetical remarks in the text of 
the algorithm itself. 

This approach to proving the correctness of algorithms has another aspect 
that is even more important: It mirrors the way we understand an algorithm. 
Recall that in Section 1.1 the reader was cautioned not to expect to read an 
algorithm like part of a novel; one or two trials of the algorithm on some sample 
data were recommended. This was done expressly because an example run- 
through of the algorithm helps a person formulate the various assertions mentally. 
It is the contention of the author that we really understand why an algorithm is 
valid only when we reach the point that our minds have implicitly filled in all the 
assertions, as was done in Fig. 4. This point of view has important psychological 
consequences for the proper communication of algorithms from one person to 
another: It implies that the key assertions, those that cannot easily be derived 
by an automaton, should always be stated explicitly when an algorithm is being 
explained to someone else. When Algorithm E is being put forward, assertion 
A 6 should be mentioned too. 

An alert reader will have noticed a gaping hole in our last proof of Algo- 
rithm E, however. We never showed that the algorithm terminates; all we have 
proved is that if it terminates, it gives the right answer! 

(Notice, for example, that Algorithm E still' makes sense if we allow its 
variables m, n, c, d, and r to assume values of the form u + v \/2, where u 
and v are integers. The variables q, a, b, a', h' are to remain integer-valued. 
If we start the method with m — 12 — 6 \[2 and n = 20 — 10 \/2, say, it will 
compute a “greatest common divisor” d = 4 — 2 \/2 with a — +2, b = — 1. Even 
under this extension of the assumptions, the proofs of assertions Al through A 6 
remain valid; therefore all assertions are true throughout any execution of the 
procedure. But if we start out with m — 1 and n — \[2, the computation never 
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terminates (see exercise 12). Hence a proof of assertions A1 through A6 does 
not logically prove that the algorithm is finite.) 

Proofs of termination are usually handled separately. But exercise 13 shows 
that it is possible to extend the method above in many important cases so that 
a proof of termination is included as a by-product. 

We have now twice proved the validity of Algorithm E. To be strictly logical, 
we should also try to prove that the first algorithm in this section, Algorithm I, 
is valid; in fact, we have used Algorithm I to establish the correctness of any 
proof by induction. If we attempt to prove that Algorithm I works properly, 
however, we are confronted with a dilemma — we can’t really prove it without 
using induction again! The argument would be circular. 

In the last analysis, every property of the integers must be proved using 
induction somewhere along the line, because if we get down to basic concepts, the 
integers are essentially defined by induction. Therefore we may take as axiomatic 
the idea that any positive integer n either equals 1 or can be reached by starting 
with 1 and repetitively adding 1; this suffices to prove that Algorithm I is valid. 
[For a rigorous study of fundamental concepts about the integers, see the article 
“On Mathematical Induction” by Leon Henkin, AMM 67 (1960), 323-338.] 

The idea behind mathematical induction is thus intimately related to the 
concept of number. The first European to apply mathematical induction to 
rigorous proofs was the Italian scientist Francesco Maurolico, in 1575. Pierre 
de Fermat made further improvements, in the early 17th century; he called 
it the “method of infinite descent.” The notion also appears clearly in the 
later writings of Blaise Pascal (1653). The phrase “mathematical induction” 
apparently was coined by A. De Morgan in the early nineteenth century. [See 
The Penny Cyclopaedia 11 (1838), 465-466; AMM 24 (1917), 199-207; 25 
(1918), 197-201; Arch. Hist. Exact Sci. 9 (1972), 1-21.] Further discussion of 
mathematical induction can be found in G. Polya’s book Induction and Analogy 
in Mathematics (Princeton, N.J.: Princeton University Press, 1954), Chapter 7. 

The formulation of algorithm-proving in terms of assertions and induction, 
as given above, is essentially due to R. W. Floyd. He pointed out that a semantic 
definition of each operation in a programming language can be formulated as a 
logical rule that tells exactly what assertions can be proved after the operation, 
based on what assertions are true beforehand [see “Assigning Meanings to Pro- 
grams,” Proc. Symp. Appl. Math., Amer. Math. Soc., 19 (1967), 19-32], Similar 
ideas were voiced independently by Peter Naur, BIT 6 (1966), 310-316, who 
called the assertions “general snapshots.” An important refinement, the notion 
of “invariants,” was introduced by C. A. R. Hoare; see, for example, CACM 14 
(1971), 39-45. Later authors found it advantageous to reverse Floyd’s direction, 
going from an assertion that should hold after an operation to the “weakest 
precondition” that must hold before the operation is done; such an approach 
makes it possible to discover new algorithms that are guaranteed to be correct, 
if we start from the specifications of the desired output and work backwards. 
[See E. W. Dijkstra, CACM 18 (1975), 453-457; A Discipline of Programming 
(Prentice-Hall, 1976).] 
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The concept of inductive assertions actually appeared in embryonic form in 
1946, at the same time as flow charts were introduced by H. H. Goldstine and 
J. von Neumann. Their original flow charts included “assertion boxes” that are 
in close analogy with the assertions in Fig. 4. [See John von Neumann, Collected 
Works 5 (New York: Macmillan, 1963), 91-99. See also A. M. Turing’s early 
comments about verification in Report of a Conference on High Speed Automatic 
Calculating Machines (Cambridge Univ., 1949), 67-68 and figures; reprinted 
with commentary by F. L. Morris and C. B. Jones in Annals of the History of 
Computing 6 (1984), 139-143.] 


The understanding of the theory of a routine 
may be greatly aided by providing, at the time of construction 
one or two statements concerning the state of the machine 

at well chosen points. . . . 

In the extreme form of the theoretical method 
a watertight mathematical proof is provided for the assertions. 

In the extreme form of the experimental method 
the routine is tried out on the machine with a variety of initial 
conditions and is pronounced fit if the assertions hold in each case. 

Both methods have their weaknesses. 

— A. M. TURING, Ferranti Mark I Programming Manual (1950) 


EXERCISES 

1. [05] Explain how to modify the idea of proof by mathematical induction, in case 
we want to prove some statement P(n) for all nonnegative integers — that is, for n — 0, 
1,2,... instead of for n = 1, 2, 3, . . . . 

► 2. [15} There must be something wrong with the following proof. What is it? 
“Theorem. Let a be any positive number. For all positive integers n we have 
a n ~ 1 = 1. Proof. If n = 1, a n_1 = a 1-1 = a 0 = 1. And by induction, assuming 
that the theorem is true for 1 , 2 , . . . , n, we have 

(„+p-i _ n _ a n_1 x a n_l 1 x 1 _ 1 

“ a( n- i)-i ~ 1 ~ ' 

so the theorem is true for n + 1 as well.” 

3. [18] The following proof by induction seems correct, but for some reason the 
equation for n = 6 gives | + | + I 5+20 + Ti = l on t ^ ie left-hand side, and § - | = § 
on the right-hand side. Can you find a mistake? “Theorem. 

-J_ + ^ + ... + I = 3_1 

1x2 2x3 (n — 1) x n 2 n 

Proof. We use induction on n. For n = 1, clearly 3/2 — 1/n = 1/(1 x 2); and, assuming 
that the theorem is true for n, 

1 1 1 

1 x 2 ^ (n — l)xn^nx(nll) 

= g_l+ 1 = L1 + (1-LUL_L. 

2 n n(n + 1 ) 2 n \n n+ 1 / 2 n + 1 

4. [20] Prove that, in addition to Eq. ( 3 ), Fibonacci numbers satisfy F n > 4> n ~ 2 . 
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5. [21] A prime number is an integer > 1 that has no exact divisors other than 1 
and itself. Using this definition and mathematical induction, prove that every integer 
> 1 may be written as a product of one or more prime numbers. (A prime number is 
considered to be the “product” of a single prime, namely itself.) 

6. [20] Prove that if Eqs. (6) hold just before step E4, they hold afterwards also. 

7. [23] Formulate and prove by induction a rule for the sums l 2 , 2 2 — l 2 , 3 2 — 2 2 + 1 2 , 
4 2 — 3 2 + 2 2 — l 2 , 5 2 — 4 2 + 3 2 — 2 2 + l 2 , etc. 

► 8. [ 25 ] (a) Prove the following theorem of Nicomachus (A.D. c. 100) by induction: 
l 3 = 1, 2 3 = 3 + 5, 3 3 = 7 + 9 + 11, 4 3 = 13 + 15 + 17 + 19, etc. (b) Use this result to 
prove the remarkable formula l 3 + 2 3 + ■ ■ • + n 3 = (1 + 2 + • • • + n) 2 . 

[Note: An attractive geometric interpretation of this formula, suggested by Warren 
Lushbaugh, is shown in Fig. 5; see Math. Gazette 49 (1965), 200. The idea is related 
to Nicomachus’s theorem and Fig. 3. Other “look-see” proofs can be found in books 
by Martin Gardner, Knotted Doughnuts (New York: Freeman, 1986), Chapter 16; 
J. H. Conway and R. K. Guy, The Book of Numbers (New York: Copernicus, 1996), 
Chapter 2.] 


Side = 5 + 5 + 5 + 5 + 5 + 5 = 5-(5 + l) 

Side = 5 + 4 + 3 + 2 + 1 + 1 + 2 + 3 + 4 + 5 
= 2 • (1 + 2+ ••• +5) 

Area = 4- l 2 -t-4-2-2 2 + 4- 3-3 2 +4-4-4 2 +4-5-5 2 
= 4-(l 3 + 2 3 + ••• +5 3 ) 


Fig. 5. Geometric version of exercise 8(b). 

9. [20] Prove by induction that if 0 < o < 1, then (1 — a) n > 1 — na. 

10. [M22] Prove by induction that if n > 10, then 2 n > n 3 . 

11. [M30] Find and prove a simple formula for the sum 

l 3 3 3 5 3 ( — l) n (2n + l) 3 

l 4 + 4 ~~ 3 4 + 4 + 5 4 + 4 + (2n + l) 4 +4 

12. [ M25 ] Show how Algorithm E can be generalized as stated in the text so that 
it will accept input values of the form u + v v^2, where u and v are integers, and the 
computations can still be done in an elementary way (that is, without using the infinite 
decimal expansion of \/2). Prove that the computation will not terminate, however, if 
m — 1 and n = y/2. 

► 13. [M23] Extend Algorithm E by adding a new variable T and adding the operation 
“T <— T+l” at the beginning of each step. (Thus, T is like a clock, counting the number 
of steps executed.) Assume that T is initially zero, so that assertion A1 in Fig. 4 
becomes “m > 0, n > 0, T = 0.” The additional condition “T = 1” should similarly be 
appended to A2. Show how to append additional conditions to the assertions in such a 
way that any one of Al, A2, . . . , A 6 implies T < 3n, and such that the inductive proof 
can still be carried out. (Hence the computation must terminate in at most 3 n steps.) 
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14. [50] (R. W. Floyd.) Prepare a computer program that accepts, as input, programs 
in some programming language together with optional assertions, and that attempts to 
fill in the remaining assertions necessary to make a proof that the computer program 
is valid. (For example, strive to get a program that is able to prove the validity of 
Algorithm E, given only assertions Al, A \ , and A 6. See the papers by R. W. Floyd 
and J. C. King in the IFIP Congress proceedings, 1971, for further discussion.) 

► 15. [HM28] ( Generalized induction.) The text shows how to prove statements P(n ) 
that depend on a single integer n, but it does not describe how to prove statements 
P(m, n) depending on two integers. In these circumstances a proof is often given by 
some sort of “double induction,” which frequently seems confusing. Actually, there 
is an important principle more general than simple induction that applies not only to 
this case but also to situations in which statements are to be proved about uncountable 
sets — for example, P(x) for all real x. This general principle is called well-ordering. 
Let “A” be a relation on a set S , satisfying the following properties: 

i) Given x, y, and z in S, if x A y and y A z, then x A z. 

ii) Given x and y in S, exactly one of the following three possibilities is true: x A y, 
x = y, or y A x. 

iii) If A is any nonempty subset of S, there is an element i in d with x Ay (that is, 
x A y or x = y) for all y in A. 

This relation is said to be a well-ordering of S. For example, it is clear that the positive 
integers are well-ordered by the ordinary “less than” relation, <. 

a) Show that the set of all integers is not well-ordered by <. 

b) Define a well-ordering relation on the set of all integers. 

c) Is the set of all nonnegative real numbers well-ordered by <? 

d) ( Lexicographic order.) Let S be well-ordered by -<, and for n > 0 let T n be the 
set of all n-tuples ( xi,X 2 , . . . ,x n ) of elements Xj in S. Define ( xi,x 2 , . . . ,x n ) A 
(yi,V 2 , • • • , Vn) if there is some k, 1 < k < n, such that Xj = yj for 1 < j < k, but 
Xk A yk in S. Is A & well-ordering of T„ ? 

e) Continuing part (d), let T = U„>i T n ; define (xi,x 2 , ■ . .,x m ) A (yi,y 2 , ■ ■ ■ , y n ) if 
Xj — yj for 1 < j < k and Xk A yk, for some k < min (m, n), or if m < n and 
Xj = yj for 1 < j < m. Is -< a well-ordering of T? 

f) Show that X is a well-ordering of S if and only if it satisfies (i) and (ii) above and 
there is no infinite sequence xi, x 2 , £ 3 , ... with Xj + 1 -< x } for all j > 1 . 

g) Let S be well-ordered by A, and let P(x) be a statement about the element x of S. 
Show that if P(x) can be proved under the assumption that P(y) is true for all 
y A x, then P(x) is true for all x in S. 

[Notes: Part (g) is the generalization of simple induction that was promised; in the case 
S = positive integers, it is just the simple case of mathematical induction treated in the 
text. In that case we are asked to prove that -P(l) is true if P(y) is true for all positive 
integers y < 1 ; this is the same as saying we should prove P(l), since P(y) certainly is 
(vacuously) true for all such y. Consequently, one finds that in many situations /'(l) 
need not be proved using a special argument. 

Part (d), in connection with part (g), gives us a powerful method of n-tuple 
induction for proving statements P{m\, . . . ,m n ) about n positive integers mi, . . . , m n . 

Part (f) has further application to computer algorithms: If we can map each state x 
of a computation into an element f(x) belonging to a well-ordered set S, in such a way 
that every step of the computation takes a state x into a state y with f(y) A f{x), then 
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the algorithm must terminate. This principle generalizes the argument about strictly 
decreasing values of n, by which we proved the termination of Algorithm 1.1E.] 


1.2.2. Numbers, Powers, and Logarithms 

Let us now begin our study of numerical mathematics by taking a good look at 
the numbers we are dealing with. The integers are the whole numbers 


..., -3, -2, -1, 0, 1, 2, 3, ... 


(negative, zero, or positive). A rational number is the ratio (quotient) of two 
integers, p/q , where q is positive. A real number is a quantity x that has a 
decimal expansion 

x — n + 0.did 2 d 3 . . . , (i) 


where n is an integer, each di is a digit between 0 and 9, and the sequence of 
digits doesn’t end with infinitely many 9s. The representation (l) means that 


d\ 


n + — + 


dk 


do. 


+ W*~ x<n + % + iw 


d 2 

10 ' loo’ 

for all positive integers k. Examples of real numbers that are not rational are 


dk 1 

10* + To*’ 


( 2 ) 


7T = 3.14159265358979 . . . , the ratio of circumference to diameter in a circle; 
4> — 1.61803398874989 . . . , the golden ratio (1 + v / 5)/2 (see Section 1.2.8). 


A table of important constants, to forty decimal places of accuracy, appears in 
Appendix A, We need not discuss the familiar properties of addition, subtrac- 
tion, multiplication, division, and comparison of real numbers. 

Difficult problems about integers are often solved by working with real 
numbers, and difficult problems about real numbers are often solved by working 
with a still more general class of values called complex numbers. A complex 
number is a quantity z of the form z = x + iy, where x and y are real and i is 
a special quantity that satisfies the equation i 2 = -1. We call x and y the real 
part and imaginary part of z , and we define the absolute value of z to be 

\z\ = \/x 2 + y 2 . ( 3 ) 

The complex conjugate of z is ~z = x — iy, and we have zz = x 2 + y 2 = |z| 2 . The 
theory of complex numbers is in many ways simpler and more beautiful than 
the theory of real numbers, but it is usually considered to be an advanced topic. 
Therefore we shall concentrate on real numbers in this book, except when real 
numbers turn out to be unnecessarily complicated. 

If u and v are real numbers with u < v, the closed interval [it . . u] is the set 
of real numbers x such that u < x < v. The open interval (u . . v) is, similarly, 
the set of x such that u < x < v. And half-open intervals [u..v) or (u . . v] are 
defined in an analogous way. We also allow u to be —00 or v to be 00 at an 
open endpoint, meaning that there is no lower or upper bound; thus (-00 . . 00 ) 
stands for the set of all real numbers, and [0 . . 00 ) denotes the nonnegative reals. 
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Throughout this section, let the letter b stand for a positive real number. If 
n is an integer, then b n is defined by the familiar rules 

b° = l, b n = b n ~ 1 b if n > 0, b n = b n+1 /b if n < 0. (4) 

It is easy to prove by induction that the laws of exponents are valid: 

b x +y = b * b y, ( b *y = b xy ^ ^ 

whenever x and y are integers. 

If u is a positive real number and if to is a positive integer, there is always 
a unique positive real number v such that v m = u; it is called the mth root of u, 
and denoted v — '{/m. 

We now define b r for rational numbers r — p/q as follows: 

feP/9 = (g) 

This definition, due to Oresme (c. 1360), is a good one, since b ap / aq = b p / q , and 
since the laws of exponents are still correct even when x and y are arbitrary 
rational numbers (see exercise 9). 

Finally, we define b x for all real values of x. Suppose first that b > 1; if x is 
given by Eq. (1), we want 

^n+di/lOH \-d k /\ 0 k < < ^n+di/lOH Mfc/10 fc + l/10 fc ^ \ 

This defines b x as a unique positive real number, since the difference between the 
right and left extremes in Eq. (7) is fr"+<h/io+-..+d fe /io t ’ (6 1 / 10 * - l); by exercise 13 
below, this difference is less than b 11 '^ (h - 1 ) / 1 0 ^ , and if we take k large enough, 
we can therefore get any desired accuracy for b x . 

For example, we find that 

1Q o. 30102999 = 1.9999999739 ... 5 1Q o. 30103000 = 2 .0000000199 . . . ; (8) 

therefore if b = 10 and x = 0.30102999 . . . , we know the value of 10 x with an 
accuracy of better than one part in 10 million (although we still don’t even know 
whether the decimal expansion of 10 x is 1.999 ... or 2.000 . . . ). 

When b < 1, we define b x = (1 /b)~ x - and when b = 1, b x = 1. With 
these definitions, it can be proved that the laws of exponents (5) hold for any 
real values of x and y. These ideas for defining b x were first formulated by John 
Wallis (1655) and Isaac Newton (1669). 

Now we come to an important question. Suppose that a positive real number 
y is given; can we find a real number x such that y = b x ? The answer is “yes” 
(provided that b ^ 1), for we simply use Eq. (7) in reverse to determine n and 
di,d 2 , . . . when b x = y is given. The resulting number x is called the logarithm 
of y to the base b , and we write this as x = log 6 y. By this definition we have 

x = b l ° 6bX = log fc (6 x ). (9) 

As an example, Eqs. (8) show that 

log 10 2 = 0.30102999 ... . ( 10 ) 
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From the laws of exponents it follows that 

\og b { x y) =log b x + \og b y, if x > 0 , y > 0 (n) 


and 

log 6 (c y ) = y log 6 c, if c > 0 . ( 12 ) 

Equation ( 10 ) illustrates the so-called common logarithms, which we get 
when the base is 10. One might expect that in computer work binary logarithms 
(to the base 2 ) would be more useful, since most computers do binary arithmetic. 
Actually, we will see that binary logarithms are indeed very useful, but not only 
for that reason; the reason is primarily that a computer algorithm often makes 
two-way branches. Binary logarithms arise so frequently, it is wise to have a 
shorter notation for them. Therefore we shall write 

lg 21 — l°g 2 x , ( x 3) 


following a suggestion of Edward M. Reingold. 

The question now arises as to whether or not there is any relationship 
between lg x and log 10 :r; fortunately there is, 

l°gio x — logio( 2 lgX ) = (lgz)(log 10 2 ), 


by Eqs. ( 9 ) and ( 12 ). Hence lgx = log 10 x/log 10 2, and in general we find that 


log c x = 


l°gfc x 

logfe C ' 


(*4) 


Equations ( 11 ), ( 12 ), and ( 14 ) are the fundamental rules for manipulating log- 
arithms. 

It turns out that neither base 10 nor base 2 is really the most conve- 
nient base to work with in most cases. There is a real number, denoted by 
e = 2.718281828459045 . . . , for which the logarithms have simpler properties. 
Logarithms to the base e are conventionally called natural logarithms, and we 
write 

lna; = log e x. ( 15 ) 

This rather arbitrary definition (in fact, we haven’t really defined e) probably 
doesn’t strike the reader as being a very “natural” logarithm; yet we’ll find that 
In x seems more and more natural, the more we work 
with it. John Napier actually discovered natural 
logarithms (with slight modifications, and without 
connecting them with powers) before the year 1590, 
many years before any other kind of logarithm was 
known. The following two examples, proved in ev- 
ery calculus text, shed some light on why Napier’s 
logarithms deserve to be called “natural”: (a) In 
Fig. 6 the area of the shaded portion is In a;, (b) If a 
bank pays compound interest at rate r, compounded 
semiannually, the annual return on each dollar is 



Fig. 6. Natural logarithm. 
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(1 + rj 2 ) 2 dollars; if it is compounded quarterly, you get (1 + r/4 ) 4 dollars; 
and if it is compounded daily you probably get (1 + r/365) 365 dollars. Now if 
the interest were compounded continuously, you would get exactly e r dollars for 
every dollar (ignoring roundoff error). In this age of computers, many bankers 
have now actually reached the limiting formula. 

The interesting history of the concepts of logarithm and exponential has 
been told in a series of articles by F. Cajori, AMM 20 (1913), 5-14, 35-47, 
75-84, 107-117, 148-151, 173-182, 205-210. 

We conclude this section by considering how to compute logarithms. One 
method is suggested immediately by Eq. ( 7 ): If we let b x — y and raise all parts 
of that equation to the 10 fc -th power, we find that 

b m < y wk < b m+1 , ( 16 ) 

for some integer m. All we have to do to get the logarithm of y is to raise y to 
this huge power and find which powers (m, rn + 1 ) of b the result lies between; 
then m/10 k is the answer to k decimal places. 

A slight modification of this apparently impractical method leads to a simple 
and reasonable procedure. We will show how to calculate log 10 x and to express 
the answer in the binary system, as 

l°gio x = n + bi /2 + 62/4 + £>3/8 + • • • . (17) 

First we shift the decimal point of x to the left or to the right so that we have 
1 < rr/lO" < 10; this determines the integer part, n. To obtain £q, 62 , . . . , we 
now set xq — x/10 n and, for k > 1 , 

h k= 0 , x k =x 2 k _ 1 , if x\_ j < 10 ; 

b k = 1 , x k =xl_ 1 /10, if x k _! > 10 . 

The validity of this procedure follows from the fact that 

1 < x k = ar 2 7 1 0 2 ' t ( n+f,1 / 2+ "' +6fc / 2 ' t ) < 10 , ( 19 ) 

for k = 0, 1 , 2, . . . , as is easily proved by induction. 

In practice, of course, we must work with only finite accuracy, so we cannot 
set x k = x k _ x exactly. Instead, we set x k — x\_ x rounded or truncated to a 
certain number of decimal places. For example, here is the evaluation of log 10 2 
rounded to four significant figures: 

xq = 2 . 000 ; 


x\ = 4.000, 

bi = 0 ; 

x 6 

= 1.845, 

^> 6—1 

X 2 = 1.600, 

&2 = 1 ; 

x 7 

= 3.404, 

67 = 0 

X 3 = 2.560, 

bz = 0 ; 


= 1.159, 

£> 8—1 

X 4 = 6.554, 

64 = 0 ; 

Xq 

= 1.343, 

69 = 0 

x 5 = 4.295, 

b$ = 1 ; 

X 10 

= 1.804, 

£>10 = 0 


Computational error has caused errors to propagate; the true rounded value of 
x 1 0 is 1.798. This will eventually cause 619 to be computed incorrectly, and 
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we get the binary value (0.0100110100010000011 . . . ) 2 , which corresponds to the 
decimal equivalent 0.301031 . . . rather than the true value given in Eq. ( 10 ). 

With any method such as this it is necessary to examine the amount of 
computational error due to the limitations imposed. Exercise 27 derives an 
upper bound for the error; working to four figures as above, we find that the 
error in the value of the logarithm is guaranteed to be less than 0.00044. Our 
answer above was more accurate than this primarily because Xo , aq, aq, and x : $ 
were obtained exactly. 

This method is simple and quite interesting, but it is probably not the 
best way to calculate logarithms on a computer. Another method is given in 
exercise 25. 


EXERCISES 

1. [00] What is the smallest positive rational number? 

2. [00] Is 1 + 0.239999999 ... a decimal expansion? 

3 . [02] What is (-3)“ 3 ? 

► 4. [05] What is (0.125) -2 / 3 ? 

5 . [05] We defined real numbers in terms of a decimal expansion. Discuss how we 
could have defined them in terms of a binary expansion instead, and give a definition 
to replace Eq. ( 2 ). 

6 . [10] Let x = m + 0.did 2 • • • and y = n + 0.eie 2 ... be real numbers. Give a rule 
for determining whether x = y, x < y, or x > y, based on the decimal representation. 

7 . [M23] Given that x and y are integers, prove the laws of exponents, starting from 
the definition given by Eq. ( 4 ). 

8 . [25] Let m be a positive integer. Prove that every positive real number u has a 
unique positive mth root, by giving a method to construct successively the values n, 
di, d 2 , . . . in the decimal expansion of the root. 

9. [M23] Given that x and y are rational, prove the laws of exponents under the 
assumption that the laws hold when x and y are integers. 

10 . [18] Prove that log 10 2 is not a rational number. 

► 11 . [10] If b = 10 and x « log 10 2, to how many decimal places of accuracy will we 

need to know the value of x in order to determine the first three decimal places of 
the decimal expansion of b x l [Note: You may use the result of exercise 10 in your 
discussion.] 

12. [02] Explain why Eq. ( 10 ) follows from Eqs. ( 8 ). 

► 13 . [M23] (a) Given that x is a. positive real number and n is a positive integer, prove 
the inequality y/1 + x — 1 < x/n. (b) Use this fact to justify the remarks following ( 7 ). 

14 . [15] Prove Eq. ( 12 ). 

15 . [10] Prove or disprove: 

l°gf> x /y = logf , x — logfi 2 /> if x,y> 0 . 

16 . [00] How can log 10 x be expressed in terms of In x and In 10? 

► 17 . [05] What is lg32? log„ 7 r? lne? log b 1? log 6 (— 1)? 
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18. [10] Prove or disprove: log 8 x = \ lg x. 

► 19. [20] Ifn is an integer whose decimal representation is 14 digits long, will the value 
of n fit in a computer word with a capacity of 47 bits and a sign bit? 

20. [10] Is there any simple relation between log 10 2 and log 2 10? 

21. [15] ( Logs of logs.) Express log 6 log 6 x in terms of In In x, In In b, and In b. 

► 22. [20] (R. W. Hamming.) Prove that 

lgz « In a: + log 10 x, 

with less than 1% error! (Thus a table of natural logarithms and of common logarithms 
can be used to get approximate values of binary logarithms as well.) 

23. [M25] Give a geometric proof that In xy = In x + In y, based on Fig. 6. 

24. [15] Explain how the method used for calculating logarithms to the base 10 at 
the end of this section can be modified to produce logarithms to base 2. 

25. [22] Suppose that we have a binary computer and a number x, 1 < x < 2. 
Show that the following algorithm, which uses only shifting, addition, and subtraction 
operations proportional to the number of places of accuracy desired, may be used to 
calculate an approximation to y = log 6 x: 

LI. [Initialize.] Set y <- 0, 2 <- x shifted right 1, k 4- 1. 

L2. [Test for end.] If x = 1, stop. 

L3. [Compare.] If x - z < 1, set z 4- z shifted right 1, k <- k + 1, and repeat this 
step. 

L4. [Reduce values.] Set x 4- x — z, z <— x shifted right k, y 4- y+ log 6 (2 fc /(2 fc — 1)), 
and go to L2. | 

[Notes: This method is very similar to the method used for division in computer 
hardware. The idea goes back in essence to Henry Briggs, who used it (in decimal 
rather than binary form) to compute logarithm tables, published in 1624. We need 
an auxiliary table of the constants log 6 2, log b (4/3), log b (8/7), etc., to as many values 
as the precision of the computer. The algorithm involves intentional computational 
errors, as numbers are shifted to the right, so that eventually x will be reduced to 1 
and the algorithm will terminate. The purpose of this exercise is to explain why it will 
terminate and why it computes an approximation to log;, x. ] 

26. [M27] Find a rigorous upper bound on the error made by the algorithm in the 
previous exercise, based on the precision used in the arithmetic operations. 

► 27. [M25] Consider the method for calculating log 10 x discussed in the text. Let x' k 
denote the computed approximation to x k , determined as follows: x(l - J) < 10 n Xo < 
x(l + e); and in the determination of x' k by Eqs. (18), the quantity y k is used in place 
of 04-i) 2 , where (4 -i) 2 (1 - 6) < y k < ( x ^^(l + e) and 1 < y k < 100. Here 5 
and e are small constants that reflect the upper and lower errors due to rounding or 
truncation. If log x denotes the result of the calculations, show that after k steps we 
have 

log 10 x + 2 log 10 (1 -5)- l/2 k < log' x < log 10 x + 2 log 10 (l + e). 

28. [M30] (R. Feynman.) Develop a method for computing If when 0 < x < 1, using 
only shifting, addition, and subtraction (similar to the algorithm in exercise 25), and 
analyze its accuracy. 
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29. [ HM20 ] Let ibea real number greater than 1. (a) For what real number b > 1 is 
b log 6 x a minimum? (b) For what integer b > 1 is it a minimum? (c) For what integer 
b > 1 is (6 + 1) log(, x a minimum? 

30. [12] Simplify the expression (lnn) lnn/lnlnn , assuming that n > 1. 

1.2.3. Sums and Products 

Let ai, d 2 , . . . be any sequence of numbers. We are often interested in sums such 

as ai + a 2 H h a n , and this sum is more compactly written using either of the 

following equivalent notations: 

n 

Y a i or Y a i- ( 1 ) 

j = 1 1 < n 

If n is zero, the value of aj + a 2 -\ h a n — )T)” =1 a j = Si <j< n a i defined to 

be zero. In general, if R(j) is any relation involving j, the symbol 

Y a i (2) 

RU) 

means the sum of all aj where j is an integer satisfying the condition R(j). If 
no such integers exist, notation ( 2 ) denotes zero. The letter j in ( 1 ) and ( 2 ) 
is a dummy index or index variable, introduced just for the purposes of the 
notation. Symbols used as index variables are usually the letters i, j, k, m, n, r, 
s , t (occasionally with subscripts or accent marks). Large summation signs like 
those in ( 1 ) and ( 2 ) can also be rendered more compactly as Y^= 1 a i or Sfi(j) a :i ■ 
The use of a and index variables to indicate summation with definite limits 
was introduced by J. Fourier in 1820. 

Strictly speaking, the notation Y^i<j< n a j ambiguous, since it does not 
clarify whether the summation is taken with respect to j or to n. In this 
particular case it would be rather silly to interpret it as a sum on values of n> j; 
but meaningful examples can be constructed in which the index variable is not 
clearly specified, as in J2j<k i£+-k)' suc ^ 1 cases the context must make clear 
which variable is a dummy variable and which variable has a significance that 
extends beyond its appearance in the sum. A sum such as Ylj<k wou ld 

presumably be used only if either j or k (not both) has exterior significance. 

In most cases we will use notation ( 2 ) only when the sum is finite — that is, 
when only a finite number of values j satisfy R(j) and have aj / 0. If an infinite 
sum is required, for example 

OO 

^ ) aj = ^ ) aj — di + U2 + «3 + • • • 

j=i j> 1 

with infinitely many nonzero terms, the techniques of calculus must be employed; 
the precise meaning of ( 2 ) is then 



0<j<n —n<j<0 
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provided that both limits exist. If either limit fails to exist, the infinite sum is 
divergent-, it does not exist. Otherwise it is convergent. 

When two or more conditions are placed under the E sign, as in ( 3 ), we 
mean that all conditions must hold. 

Four simple algebraic operations on sums are very important, and familiarity 
with them makes the solution of many problems possible. We shall now discuss 
these four operations. 

a) The distributive law, for products of sums: 


V «(») 7 K S(j) / R(i) 'S(j) 

To understand this law, consider for example the special case 

a i ) ( J = ( a i + a-2){bi + b 2 + 63) 

i = 1 / \j=l / 

= (oi&l + ni&2 + 0163) + (a2&l + 02^2 + ^2^3) 

i= 1 \j = 1 / 


(4) 


It is customary to drop the parentheses on the right-hand side of ( 4 ); a double 
summation such as Efl(i)(Es(j) a,,') is written simply J2 R (i) E s(j) a >r 

b) Change of variable: 


Yl ai - a i ~ a Ptt)- (5) 

m RU) R(pU)) 

This equation represents two kinds of transformations. In the first case we are 
simply changing the name of the index variable from i to j. The second case 
is more interesting: Here p(j) is a function of j that represents a permutation 
of the relevant values; more precisely, for each integer i satisfying the relation 
R{i), there must be exactly one integer j satisfying the relation p(j) = i. This 
condition is always satisfied in the important cases p(j) — c + j and p(j) = c-j, 
where c is an integer not depending on j, and these are the cases used most 
frequently in applications. For example, 

a j = H “j- 1 = a 3~ 1 ' ( 6 ) 

l<j<n l<j-l<n 2 <j<n+l 

The reader should study this example carefully. 

The replacement of j by p(j) cannot be done for all infinite sums. The 
operation is always valid if p(j) = c ± j, as above, but in other cases some 
care must be used. [For example, see T. M. Apostol, Mathematical Analysis 
(Reading, Mass.: Addison- Wesley, 1957), Chapter 12 . A sufficient condition to 
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guarantee the validity of (5) for any permutation of the integers, p(j), is that 
E R(j) M exists.] 

c) Interchanging order of summation: 

y y a n ~ y y °d- m 

S(j) S(j) R(i) 

Let us consider a very simple special case of this equation: 

2 

'y 1 ^ ' a ij = y 1 ( a il + a i2)) 

R(i) j = l R(i) 

2 

yi y a ij = a »i + y a *2- 

j = l R(i) R(i) R(i) 

By Eq. (7), these two are equal; this says no more than 

y {bi +Ci) = J2 b i + y c *> (8) 

R(i ) ^(*) -^(*) 

where we let 6; = a^i and Ci — a^. 

The operation of interchanging the order of summation is extremely useful, 
since it often happens that we know a simple form for E R(i) a iji n °t for 
E s(j) a ij- W e frequently need to interchange the summation order also in a 
more general situation, where the relation S(j) depends on i as well as j. In such 
a case we can denote the relation by u S(i,j). n The interchange of summation 
can always be carried out, in theory at least, as follows: 

y y a v ~ y y a *j> (9) 

R(i) S(i,j) S’(j) R'(i,j) 

where S'(j ) is the relation “there is an integer i such that both R(i) and S(i,j ) 
are true”; and R'(i,j) is the relation “both R(i) and S(i,j) are true.” For 
example, if the summation is X]"=i E}=i a ij> then S'(j) is the relation “there is 
an integer i such that 1 < i < n and 1 < j < if that is, 1 < j < n; and R'{i,j) 
is the relation “1 < i < n and 1 < j < if that is, j < i < n. Thus, 

n i n n 

y y«b- = y y. q d • ( iq ) 

*= 1 j — 1 j=l i=j 

[Note: As in case (b), the operation of interchanging order of summation is not 
always valid for infinite series. If the series is absolutely convergent — that is, if 
E R(i) E s(j) I a ij I exists — it can be shown that Eqs. (7) and (9) are valid. Also 
if either one of R(i) or S(j) specifies a finite sum in Eq. (7), and if each infinite 
sum that appears is convergent, then the interchange is justified. In particular, 
Eq. (8) is always true for convergent infinite sums.] 
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d) Manipulating the domain. If R(j) and S(j) are arbitrary relations, we 

have 

^2 IX a i = 5Z a i + ^2 a j' ( 1:L ) 

R(j) S(j) R(j) or S(j) R{j) and S(j) 

For example, 

^2 a i + X ] a i ~ ( ^2 a i J + flm ’ ( 12 ) 

1 ' 1 < j < n ' 

assuming that 1 < m < n. In this case u R(j) and S(j)” is simply “j = m,” so we 
have reduced the second sum to simply “a m .” In most applications of Eq. (n), 
either R(j) and S(j) are simultaneously satisfied for only one or two values of j , 
or else it is impossible to have both R(j) and S(j) true for the same j. In the 
latter case, the second sum on the right-hand side of Eq. (11) simply disappears. 

Now that we have seen the four basic rules for manipulating sums, let’s 
study some further illustrations of how to apply these techniques. 

Example 1. 

X a 3 = X a 3 + X a i by rule ( d ) 

0<j<n 0<j<n 0<j<n 

j even j odd 

= X a2 i + X a2 i +1 by rule (b) 

0<2j<n 0<2j + l<n 

2 j even 2j+lodd 

= X a2 i + X a 2j+ 1 ' 

0<j<n/2 0<j<n/2 

The last step merely consists of simplifying the relations below the ]T’s. 

Example 2. Let 


n i n n 


Si — ^22 a * a f — 5 2 ^ 2 aia 3 

i = 0 j = 0 j = 0 i=j 

by rule (c) [see Eq. (10)] 

n n 

— ^2 X aia j 

i= 0 j=i 

by rule (b), 


interchanging the names i and j and recognizing that a l a l = . If we denote 

the latter sum by S2, we have 



by Eq. (8) 


by rule (d) 

[see Eq. (12)] 
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^ ^ ^ ^ di&j ^ ^ di&i 
2=0 j = 0 


by Eq. (8) 


2=0 


= ( E a * ) ( J2 a i ) + ( E a * ) rule ( a ) 

^ 2=0 ' \j=0 


2=0 


n \ 2 


2 = 0 


= E a 0 + E a * 


2 = 0 


by rule (b). 


Thus we have derived the important identity 


v- v- 1 ^ V 


2= 0 j =0 


2 = 0 


2=0 


(13) 


Example 3 ( The sum of a geometric progression). Assume that x / 1, n > 0. 
Then 


a + ax + • • • + ax n = 


ax J 


0<j<n 

— a + ax i 

l<j<n 

= a + x ax^ 1 

l<j<n 


— a + x 


ax J 


by definition ( 2 ) 
by rule (d) 

by a very special case of (a) 
by rule (b) [see Eq. (6)] 


0 < j < n — 1 


— a + x ax^ — ax n+1 by rule (d). 

0<j<n 

Comparing the first relation with the last, we have 


(1 — X) n'rJ 

0<jf<n 

hence we obtain the basic formula 


ax J = d — dx 


n+ 1 . 




(14) 


0<j<n 

Example 4 (The sum of an arithmetic progression). Assume that n > 0. Then 
a + (a+b) + • • • + (a + nb) 

= (a + bj) by definition ( 2 ) 

0<j<n 
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= X {a + b( n ~ j)) by rule (b) 

0 <n—j<n 

= (a + bn — bj ) by simplification 

0<j<n 

= X ( 2a + 6n ) ~ X ( a + b y Ec t- ( 8 ) 

0<j<n 0<j<n 


= (n + l)(2a + bn) - X (a + h/), 

0<j<n 

since the first sum simply adds together (n + 1) terms that do not depend on j. 
Now by equating the first and last expressions and dividing by 2 , we obtain 

X ( a + b j) = a(n+ 1) + \bn(n+ 1). (15) 

0<j<n 


This is n + 1 times | (a + (a + bn )) , which can be understood as the number of 
terms times the average of the first and last terms. 

Notice that we have derived the important equations (13), (14), and (15) 
purely by using simple manipulations of sums. Most textbooks would s im ply 
state those formulas, and prove them by induction. Induction is, of course, a 
perfectly valid procedure; but it does not give any insight into how on earth 
a person would ever have dreamed the formula up in the first place, except by 
some lucky guess. In the analysis of algorithms we are confronted with hundreds 
of sums that do not conform to any apparent pattern; by manipulating those 
sums, as above, we can often get the answer without the need for ingenious 
guesses. 

Many manipulations of sums and other formulas become considerably sim- 
pler if we adopt the following bracket notation: 


[statement] = | q’ 


Then we can write, for example, 


if the statement is true; 
if the statement is false. 


X a i = X a i [^(i)] » 

R(i) i 


(16) 


(17) 


where the sum on the right is over all integers j, because the terms of that 
infinite sum are zero when R(j) is false. (We assume that aj is defined for all j.) 

With bracket notation we can derive rule (b) from rules (a) and (c) in an 
interesting way: 


X a pu) ~ X a pu) [-^(p(j))] 

r (pU)) i 


= EX> [^w] [*=p(j)] 

3 * 


1.2.3 


SUMS AND PRODUCTS 


33 


= ^a<[i?(i)] Xt* =p(j)] ■ (18) 

* 3 


The remaining sum on j is equal to 1 when R(i) is true, if we assume that p is 
a permutation of the relevant values as required in (5); hence we are left with 
a i[-R(*)]> which is Sij(i) a i- This proves (5). If p is not such a permutation, 
(18) tells us the true value of S«( p (j)) a p(j)- 

The most famous special case of bracket notation is the so-called Kronecker 
delta symbol, 


A -j 



if i = j, 

if * / j, 


(T9) 


introduced by Leopold Kronecker in 1868. More general notations such as (16) 
were introduced by K. E. Iverson in 1962; therefore (16) is often called Iverson’s 
convention. [See D. E. Knuth, AMM 99 (1992), 403-422.] 

There is a notation for products, analogous to our notation for sums: The 
symbols 

II a i ( 20 ) 

fl(i) 


stand for the product of all aj for which the integer j satisfies R(j). If no such 
integer j exists, the product is defined to have the value 1 ( not 0). 

Operations (b), (c), and (d) are valid for the [[[[-notation as well as for the 
^-notation, with suitable simple modifications. The exercises at the end of this 
section give a number of examples of product notation in use. 

We conclude this section by mentioning another notation for multiple sum- 
mation that is often convenient: A single y]-sign may be used with one or 
more relations in several index variables, meaning that the sum is taken over all 
combinations of variables that meet the conditions. For example, 


X X a T “ X a T ’ X X °'T — X “tr 

0 <i<n 0 <j<n 0 <i,j<n 0 <i<n 0 <j<i 0<j<i<n 

This notation gives no preference to one index of summation over any other, so 
it allows us to derive (10) in a new way: 


n i 

XX“, = X a»j[ 1 < * < n][l < j < i] — E aij[l <j <n][j <i<n\ 

i = 1 3 — 1 i,j i,j 

n n 

= X X a T » 

3 = 1 *=j 

using the fact that [1 < i < n][l < j < i] = [1 < j < i < n] = [1 < j < n][j < i < n]. 
The more general equation (9) follows in a similar way from the identity 

[#(*)] [S(i,j)] = [R(i) and S(i,j)] = [5'(j)] j)] . ( 21 ) 
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A further example that demonstrates the usefulness of summation with 
several indices is 

! ( 22 ) 

JlH \-jn=n 

3 1 > ' • • > Jn >0 

where a is an n-tuply subscripted variable; for example, if n = 5 this notation 
stands for 

a lllll + 021110 + 022100 + 031100 + O3200O + O41000 + 050000- 

(See the remarks on partitions of a number in Section 1.2.1.) 


EXERCISES — First Set 

1. [01] What does the notation X)i<j< n a i mean , if n = 3.14? 

2. [10] Without using the ^-notation, write out the equivalent of 


and also the equivalent of 


2n + 1 ’ 


A-' 2n 2 + 1 ' 

0<n 2 <5 

► 3. [13] Explain why the two results of the previous exercise are different, in spite of 
rule (b). 

4. [10] Without using the ]U-notation, write out the equivalent of each side of 
Eq. (10) as a sum of sums for the case n = 3. 

► 5. [HM20] Prove that rule (a) is valid for arbitrary infinite series, provided that the 
series converge. 

6. [HM20] Prove that rule (d) is valid for an arbitrary infinite series, provided that 
any three of the four sums exist. 

7. [HM23] Given that c is an integer, show that J2r(j) a i = Sh(c-i) °c-j> even if 
both series are infinite. 

8. [HM25] Find an example of infinite series in which Eq. (7) is false. 

► 9. [05] Is the derivation of Eq. (14) valid even if n = —1? 

10. [05] Is the derivation of Eq. (14) valid even if n = —2? 

11. [03] What should the right-hand side of Eq. (14) be if x = 1? 

12. [10] What is 1 + 7 -f ^ + 3^5 + • • • + (7) ? 

13. [10] Using Eq. (15) and assuming that m < n, evaluate X)j =Tn j- 

14. [11] Using the result of the previous exercise, evaluate '}Zl =r jk. 

► 15. [M22] Compute the sum 1 x 2 + 2 x2 2 +3 x2 3 H bn x2 n for small values of n. Do 


you see the pattern developing in these numbers? If not, discover it by manipulations 
similar to those leading up to Eq. (14). 
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16. [M22] Prove that 

nx n+2 — (n + \)x n+x + x 

(x^rji ’ 

if x / 1, without using mathematical induction. 

► 17. [MOO] Let S be a set of integers. What is Ylj^s 1? 

18. [. M20 ] Show how to interchange the order of summation as in Eq. (9) given that 
R(i) is the relation “n is a multiple of i” and S(i,j) is the relation “1 < j < i” 

19. [20] What is X^= m ( a t — a j-i)? 

► 20. [25] Dr. I. J. Matrix has observed a remarkable sequence of formulas: 

9x1 + 2 = 11, 9x 12 + 3 = 111, 9 x 123 + 4 = 1111, 9 x 1234 + 5 = 11111. 



a) Write the good doctor’s great discovery in terms of the ^-notation. 

b) Your answer to part (a) undoubtedly involves the number 10 as base of the decimal 
system; generalize this formula so that you get a formula that will perhaps work 
in any base b. 

c) Prove your formula from part (b) by using formulas derived in the text or in 
exercise 16 above. 

► 21. [M25] Derive rule (d) from (8) and (17). 

► 22. [20] State the appropriate analogs of Eqs. (5), (7), (8), and (11) for products 
instead of sums. 

23. [10] Explain why it is a good idea to define JZru) a i and Ilfi(j) a i as zero and 
one, respectively, when no integers satisfy R(j). 

24. [20] Suppose that R(j) is true for only finitely many j . By induction on the 
number of integers satisfying R(j), prove that log 6 ]~I /?(.,) a j = J2R(j)(^°&b a i)> assuming 
that all aj > 0. 

► 25. [15] Consider the following derivation; is anything amiss? 


( 


E 




= E E 5 

l<i<n l<j<n ^ 




26. [25] Show that n"=o IIj=o °< a i ma y expressed in terms of n"=o a ' manip- 
ulating the n- no t a ti on as stated in exercise 22. 

27. [M20] Generalize the result of exercise 1.2. 1-9 by proving that 

n n 

nu-*) - 1 ~ e “j . 

3 = 1 3 = 1 

assuming that 0 < aj <1. 

28. [M22] Find a simple formula for YY^ =2 (1 ~ 1 /j 2 )- 

► 29. [M30] (a) Express 52" =0 0 a i a i a k in terms of the multiple-sum notation 

explained at the end of the section, (b) Express the same sum in terms of a ‘> 

£r=o a i< and Er=o a i I see E 9- (!3)]- 
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► 30. [ M23 ] (J. Binet, 1812.) Without using induction, prove the identity 


Y a i x i ) ( Y ) = ( Y a >Vi ) ( Y b i x ’ ) + Y ( a i bk ~ akbj)(x jVk - x kyj ). 

j= 1 ' '3= 1 ' = 1 ‘ \j'= 1 / l<j<fc<n 


[An important special case arises when W\, . . . , w n , zi, . . . , z„ are arbitrary complex 
numbers and we set aj = Wj, bj = Zj, Xj = Wj, yj = Zj: 


1 2^l«bT I l E^ 2 ) = Y w > z i + E \WjZ k -w k Zj\‘. 

\j=l / \j=:l / j = 1 1 <j<k<n 

The terms \wjZ k - are nonnegative, so the famous Cauchy-Schwarz inequality 

(Eki 2 )(Ei^i 2 )> X>* 2 

\i=l / Vj=l / J=1 

is a consequence of Binet’s formula.] 

31. [ M20 ] Use Binet’s formula to express the sum Yli<j<k<n ( u 3 ~ u k)(vj — v k ) in 
terms of £” =1 UjVj, £” =1 and ]T"=i v j- 

32. [M20] Prove that 

n m 

^ ' 0,ij = y [ Oijl . . . fflt n n . 

3 — 1 *=1 l<il <m 

► 33. [ M30 ] One evening Dr. Matrix discovered some formulas that might even be 
classed as more remarkable than those of exercise 20: 


= 0, 

= 0, 

= 1 , 

= a + b + c. 


Prove that these formulas are a special case of a general law; let xi,x 2 , ■ . . , x n be 
distinct numbers, and show that 

n / j x f O’ if 0 < r < n — 1; 

Y\ x 3 II -**))= < !> ifr = n-l; 

j = l V / i <k<n / I Xli-i x ji if r = n. 

k*j 


(a — b) (a — c ) 

+ 

(6- 

a) (6 — c) 

+ 

(c — a)(c — 6) 

a 

_ 1 _ 


b 

1 

c 

(a — b) (a — c ) 

1 

(6- 

a)(b — c) 


(c — a)(c — 6) 

a 2 



b 2 

1 

c 2 

(a — b) (a — c) 

d 

(fe- 

a) (6 — c) 

T 

(c — a) (c — b ) 

a 3 

+ 


b 3 

+ 

c 3 


n ( x o~ x k 

j=l V / l<k<n 

k^j 

34. [Af.25] Prove that 


^ lll< r <n, r *m ( X + k ~ r ) _ ^ 

k = 1 ril<r<n,7-/fc(^ — r ) 
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provided that 1 < m < n and x is arbitrary. For example, if n = 4 and m = 2, then 

x(x — 2)(x — 3) (x + l)(x — l)(x — 2) (x + 2)x(x — 1) (x + 3)(a: + 1)® 

(-l)(-2)(— 3) + (1)( 1)(— 2) + (2)(1)( — 1) (3)(2)(1) 

35. [HM20] The notation sup fl yj a,j is used to denote the least upper bound of the 
elements a,j, in a manner exactly analogous to the J2~ an d n~ n °t a ti°ns. (When R(j) 
is satisfied for only finitely many j, the notation maxjt(j) a : is often used to denote the 
same quantity.) Show how rules (a), (b), (c), and (d) can be adapted for manipulation 
of this notation. In particular discuss the following analog of rule (a): 

( su Pj?(i) a«) + ( su Psu) b i) = su PH(i)( su Ps(j)( a i + 
and give a suitable definition for the notation when R(j) is satisfied for no j. 


EXERCISES — Second Set 


Determinants and matrices. The following interesting problems are for the reader 
who has experienced at least an introduction to determinants and elementary matrix 
theory. A determinant may be evaluated by astutely combining the operations of: (a) 
factoring a quantity out of a row or column; (b) adding a multiple of one row (or 
column) to another row (or column); (c) expanding by cofactors. The simplest and 
most often used version of operation (c) is to simply delete the entire first row and 
column, provided that the element in the upper left corner is +1 and the remaining 
elements in either the entire first row or the entire first column are zero; then evaluate 
the resulting smaller determinant. In general, the cofactor of an element ay in an 
n x n determinant is (— l) t+: * times the (n — 1) X (n — 1) determinant obtained by 
deleting the row and column in which aij appeared. The value of a determinant is 
equal to a ij ' cofactor (ay) summed with either i or j held constant and with the 
other subscript varying from 1 to n. 

If (by) is the inverse of matrix (aij), then by equals the cofactor of aji ( not aij), 
divided by the determinant of the whole matrix. 

The following types of matrices are of special importance: 


Vandermonde’s matrix, 
an = x) 


Combinatorial matrix, 
a^j — y + SijX 


/X\ 

X 2 ■ 



/x + y 

y 

■ y \ 

x\ 

x 2 ■ 

• x l 


y 

x + y . 

y 

w 


• ■ x n J 


V y 

y 

. x + y) 


Cauchy’s matrix, 
an = 1 /(xi + yj) 

/i/(xi+yi) 1 / (xi + ?/2 ) 1 / (xi + 2/n)^ 

i/(x2 + yi) l/(x 2 + y 2 ) ■■■ l/(x 2 +y n ) 


\l/(x n +yi) 1/ (x n T y 2 ) ... 1 /(x n +y n )' 
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36. ' M23[ Show that the determinant of the combinatorial matrix is x n 1 (.r ny). 

► 37. [M 24 } Show that the determinant of Vandermonde’s matrix is 

n n ( x 3~xi). 

l<j<n l<i<j<n 

► 38. [M25] Show that the determinant of Cauchy’s matrix is 


(xj Xi)(yj yi) j {xi + yj). 

39. [M23] Show that the inverse of a combinatorial matrix is a combinatorial matrix 
with the entries b tj = (- y + 5ij ( x + ny))/x(x + ny). 

40. [M24] Show that the inverse of Vandermonde’s matrix is given by 


y (-i) j i xk 1 ...x kri 


l<fci <-"<fc n _j- < n 
k l 


1 J ( x k - Xi). 


Don’t be dismayed by the complicated sum in the numerator — it is just the coefficient 
of x J_1 in the polynomial (xi - x) . . . ( x n - x)/(xi - x). 

41. [M26] Show that the inverse of Cauchy’s matrix is given by 
bi i = ( n ( x f +yh)(xk + 2/i)'j j (xj +yi)( ]^[ (xj - 


42. [ M18 ] What is the sum of all 1 1 ~ elements in the inverse of the combinatorial 
matrix? 


43. [M24 ] What is the sum of all n 2 elements in the inverse of Vandermonde’s matrix? 
[Hint: Use exercise 33.] 

► 44. [ M26 ] What is the sum of all n 2 elements in the inverse of Cauchy’s matrix? 

► 45. [M25] A Hilbert matrix, sometimes called an nxn segment of the (infinite) Hilbert 
matrix, is a matrix for which aij = l/(i + j- 1). Show that this is a special case of 
Cauchy’s matrix, find its inverse, show that each element of the inverse is an integer, 
and show that the sum of all elements of the inverse is n 2 . [Note: Hilbert matrices 
have often been used to test various matrix manipulation algorithms, because they 
are numerically unstable, and they have known inverses. However, it is a mistake 
to compare the known inverse, given in this exercise, to the computed inverse of a 
Hilbert matrix, since the matrix to be inverted must be expressed in rounded numbers 
beforehand; the inverse of an approximate Hilbert matrix will be somewhat different 
from the inverse of an exact one, due to the instability present. Since the elements 
of the inverse are integers, and since the inverse matrix is just as unstable as the 
original, the inverse can be specified exactly, and one could try to invert the inverse. 
The integers that appear in the inverse are, however, quite large.] The solution to this 
problem requires an elementary knowledge of factorials and binomial coefficients, which 
are discussed in Sections 1.2.5 and 1.2.6. 


► 46. [M30] Let A be an m x n matrix, and let 6 be an n x m matrix. Given that 
1 < Ji»j 2 , • . . ,jm < n, let Aj 1 j 2 ...j m denote the mx m matrix consisting of columns 
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j i, . . . , jm of A, and let Bj 1 j 2 ...j m denote the mxm matrix consisting of rows ji, . . ■ ,j m 
of B. Prove the Binet-Cauchy identity 

det (AB) = det(A jlj2 ... jm )det(B jlj2 ... jm ). 

l<jl <j 2 <--<jm<n 

(Note the special cases: (i) m = n, (ii) m = 1, (iii) B = A T , (iv) m > n, (v) m = 2.) 
47. [M27] (C. Krattenthaler.) Prove that 

( (x + ^Xz + gs) (x +pi)(x + q 3 ) (x + pi)(x + p 2 )\ 

(y + Q 2 ){y + qs) (y + pi)(y + q3) (y + pi)(y + P 2 ) 

{z + q 2 )(z + q 3 ) ( z + p!)(z + q 3 ) (z + pi) (z + p 2 ) J 

= (x - y)(x - z)(y - z)(p! - q 2 )(pi - q 3 )(p 2 - qa), 

and generalize this equation to an identity for an nxn determinant in 3n — 2 variables 
xi, . . . , x„, pi , . . . , p n - 1 , q 2 , , q n . Compare your formula to the result of exercise 38. 


1.2.4. Integer Functions and Elementary Number Theory 

If x is any real number, we write 

(xj = the greatest integer less than or equal to x (the floor of x); 

[x] = the least integer greater than or equal to x (the ceiling of x). 

The notation [x] was often used before 1970 for one or the other of these functions, 
usually the former; but the notations above, introduced by K. E. Iverson in the 
1960s, are more useful, because [_xj and [x] occur about equally often in practice. 
The function [zj is sometimes called the entier function, from the French word 
for “integer.” 

The following formulas and examples are easily verified: 

|y2|=l, (V2]=2, |+ij=0, [-^ 1 = 0 , L“^J = _1 ( not zer o!); 


[x] = L&J if and only if x is an integer, 

[x] = |_xj + 1 if and only if x is not an integer; 


|_— xj = — |"x] ; x — 1 < L^J < x < \x] < x + 1. 

Exercises at the end of this section list other important formulas involving the 
floor and ceiling operations. 

If x and y are any real numbers, we define the following binary operation: 


x mod y = x - y [x/y \ , if y ± 0; 


x mod 0 = x. 


From this definition we can see that, when j/ / 0, 

x mod y 


0 < 


x 

2/J 


< 1 . 


y 


Consequently 

a) if y > 0, then 0 < x mod y < y\ 

b) if y < 0, then 0 > x mod y > y; 


(1) 

( 2 ) 
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c) the quantity x — (x mod y) is an integral multiple of y. 

We call xmody the remainder when x is divided by y ; similarly, we call [x/y\ 
the quotient. 

When x and y are integers, “mod” is therefore a familiar operation: 

5 mod 3 = 2, 18 mod 3 = 0, —2 mod 3 = 1. (3) 

We have x mod y = 0 if and only if a: is a multiple of y, that is, if and only if x is 
divisible by y. The notation y\x. read “y divides x ,” means that y is a positive 
integer and x mod y — 0. 

The “mod” operation is useful also when x and y take arbitrary real values. 
For example, with trigonometric functions we can write 

tan x — tan ( x mod ir) . 

The quantity x mod 1 is the fractional part of x; we have, by Eq. (1), 

x = [x\ + (x mod 1). (4) 

Writers on number theory often use the abbreviation “mod” in a different 
but closely related sense. We will use the following form to express the number- 
theoretical concept of congruence: The statement 

x = y (modulo 2) (5) 

means that x mod z — y mod 2; it is the same as saying that x — y is an integral 
multiple of 2. Expression (5) is read, “x is congruent to y modulo 2.” 

Let’s turn now to the basic elementary properties of congruences that will 
be used in the number-theoretical arguments of this book. All variables in the 
following formulas are assumed to be integers. Two integers x and y are said 
to be relatively prime if they have no common factor, that is, if their greatest 
common divisor is 1; in such a case we write x 1 y. The concept of relatively 
prime integers is a familiar one, since it is customary to say that a fraction is in 
“lowest terms” when the numerator is relatively prime to the denominator. 

Law A. If a = b and x = y, then a ± x = b ± y and ax = by (modulo m). 

Law B. If ax = by and a — b, and if a _L to, then x = y (modulo m). 

Law C . a = b (modulo m ) if and only if an = bn (modulo mn ), when n / 0. 

Law D. If r X s, then a = b (modulo rs ) if and only if a = b (modulo r) and 
a = b (modulo s). 

Law A states that we can do addition, subtraction, and multiplication 
modulo m just as we do ordinary addition, subtraction, and multiplication. 
Law B considers the operation of division and shows that, when the divisor 
is relatively prime to the modulus, we can also divide out common factors. Laws 
C and D consider what happens when the modulus is changed. These laws are 
proved in the exercises below. 

The following important theorem is a consequence of Laws A and B. 
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Theorem F ( Fermat’s theorem, 1640). If p is a prime number, then a p = a 
(modulo p) for all integers a. 

Proof. If a is a multiple of p, obviously a p = 0 = a (modulo p). So we need 
only consider the case a mod p / 0. Since p is a prime number, this means that 
alp. Consider the numbers 

Omodp, amodp, 2 a mod p, ..., (p — ljomodp. ( 6 ) 

These p numbers are all distinct, for if ax mod p — ay mod p, then by defini- 
tion ( 5 ) ax = ay (modulo p); hence by Law B , x = y (modulo p). 

Since ( 6 ) gives p distinct numbers, all nonnegative and less than p, we see 
that the first number is zero and the rest are the integers 1 , 2 , . . . , p — 1 in some 
order. Therefore by Law A, 

(a)(2a) . . . ((p — l)a) = 1 • 2 . . . (p — 1 ) (modulo p). ( 7 ) 

Multiplying each side of this congruence by a, we obtain 

a p (l • 2 . . . (p — 1 )) = a(l • 2 . . . (p — 1 )) (modulo p); ( 8 ) 

and this proves the theorem, since each of the factors 1 , 2 , . . . , p — 1 is relatively 
prime to p and can be canceled by Law B. | 

EXERCISES 

1 . [00] What are [LlJ, [- 1 . 1 J, T-l.ll, |0.99999J, and [lg35j? 

► 2. [01] What is [>.11? 

3. [M10] Let n be an integer, and let a; be a real number. Prove that 
a) [xj < n if and only if x < n; b) n < [x\ if and only if n < x; 

c) M < ri if and only if x < n; d) n < [x] if and only if n < x; 

e) [xj = n if and only if x — 1 < n < x, and if and only if n < x < n + 1 ; 

f) [~x] = n if and only if x < n < x + 1 , and if and only if n — 1 < x < n. 

[These formulas are the most important tools for proving facts about [xj and [x] .] 

► 4. [M10] Using the previous exercise, prove that [—x] = — [x]. 

5. [16] Given that x is a positive real number, state a simple formula that expresses 
x rounded to the nearest integer. The desired rounding rule is to produce [xj when 
x mod 1 < 5 , and to produce [x] when x mod 1 > \. Your answer should be a single 
formula that covers both cases. Discuss the rounding that would be obtained by your 
formula when x is negative. 

► 6 . [20] Which of the following equations are true for all positive real numbers x? 

(a) = [Vx\-, (b) rvT^Tl = fv^l; (c) ["vTaTI = rv*i- 

7. [M15] Show that [xj + [y\ < |_x + y\ and that equality holds if and only if 
x mod 1 + y mod 1 < 1. Does a similar formula hold for ceilings? 

8 . [00] What are 100 mod 3, 100 mod 7, —100 mod 7, —100 mod 0? 

9. [05] What are 5 mod —3, 18 mod —3, —2 mod —3? 

► 10. [10] What are 1.1 mod 1, 0.11 mod .1, 0.11 mod —.1? 

11. [00] What does “x = y (modulo 0)” mean by our conventions? 
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12. [00] What integers are relatively prime to 1? 

13. [MOO] By convention, we say that the greatest common divisor of 0 and n is |n|. 
What integers are relatively prime to 0? 

► 14. [12] If x mod 3 = 2 and x mod 5 = 3, what is x mod 15? 

15. [10] Prove that z(x mod y) = (zx) mod (zy). [Law C is an immediate consequence 
of this distributive law.] 

16. [M10] Assume that y > 0. Show that if (x — z)/y is an integer and if 0 < z < y, 
then z = x mod y. 

17. [M15] Prove Law A directly from the definition of congruence, and also prove half 
of Law D: If a = b (modulo rs), then a = b (modulo r ) and a = b (modulo s). (Here r 
and s are arbitrary integers.) 

18. [M15] Using Law B, prove the other half of Law D: If a = b (modulo r) and a = b 
(modulo s), then a = b (modulo rs), provided that rls. 

► 19. [M10] ( Law of inverses.) If n _L to, there is an integer n! such that nn! = 1 
(modulo m). Prove this, using the extension of Euclid’s algorithm (Algorithm 1.2. IE). 

20. [M15] Use the law of inverses and Law A to prove Law B. 

21. [M22] ( Fundamental theorem of arithmetic.) Use Law B and exercise 1.2. 1-5 to 
prove that every integer n > 1 has a unique representation as a product of primes 
(except for the order of the factors). In other words, show that there is exactly one 
way to write n = P1P2 ■ ■ - Pk, where each pj is prime and Pi < P2 < • • • < pk- 

► 22. [M10] Give an example to show that Law B is not always true if a is not relatively 
prime to m. 

23. [ M10 ] Give an example to show that Law D is not always true if r is not relatively 
prime to s. 

► 24. [M20] To what extent can Laws A, B, C, and D be generalized to apply to 
arbitrary real numbers instead of integers? 

25. [MO 2] Show that, according to Theorem F, a p_1 modp = [a is not a multiple 
of p], whenever p is a prime number. 

26. [ M15 ] Let p be an odd prime number, let a be any integer, and let b — a <p ~ 1 ^ 2 . 
Show that b mod p is either 0 or 1 or p — 1. [Hint: Consider (b + 1)(6 — 1).] 

27. [M15] Given that n is a positive integer, let pin) be the number of values among 
{0, 1, . . . ,n — 1} that are relatively prime to n. Thus <p( 1) = 1, <p(2) = 1, <p(3) = 2, 
<p( 4) = 2, etc. Show that <p(p) = p — 1 if p is a prime number; and evaluate <p(p e ), 
when e is a positive integer. 

► 28. [M25] Show that the method used to prove Theorem F can be used to prove the 
following extension, called Euler’s theorem: a = 1 (modulo m), for any positive 
integer m, when a T m. (In particular, the number n in exercise 19 may be taken to 
be n' e ^ rn ' ) ~ 1 modm.) 

29. [M22] A function f(n) of positive integers n is called multiplicative if f(rs) = 
f(r)f{s) whenever r _L s. Show that each of the following functions is multiplicative: 
(a) f(n) = n c , where c is any constant; (b) /(n) = [n is not divisible by k 2 for any 
integer k > 1]; (c) /(n) = c k , where k is the number of distinct primes that divide n; 
(d) the product of any two multiplicative functions. 
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30. [ M30 ] Prove that the function <p(n) of exercise 27 is multiplicative. Using this 
fact, evaluate <p( 1000000), and give a method for evaluating <p(n) in a simple way once 
n has been factored into primes. 

31. [ M22 ] Prove that if f(n) is multiplicative, so is g(n) = Yld\n /(^)- 

32. [Ml 8] Prove the double-summation identity 

E = E E /( c ’ cd )> 

d\n c\d c\n d\(n/c) 


for any function f(x,y). 

33. [ M18 ] Given that m and n are integers, evaluate (a) |_| (n + m)J + [|(n — m + 1)] ; 
(b) [|(n + m)] + [ i(n — m + 1)] . (The special case m — 0 is worth noting.) 

► 34. [M21] What conditions on the real number 6 > 1 are necessary and sufficient to 
guarantee that [log^a:] = [log b L*J J for all real x > 1? 

► 35. [M20] Given that m and n are integers and n > 0, prove that 

L(x + m)/nj = |_([_a:J + m)/nj 


for all real x. (When m = 0, we have an important special case.) Does an analogous 
result hold for the ceiling function? 

36. [M23] Prove that Sfc=iL^/2j = L n2 /4J; also evaluate J2k= if^/2]- 
► 37. [M30] Let m and n be integers, n > 0. Show that 


E 

0 </e<n 


mk + x 
n 


(m 


!)(»-!) 

2 


+ +d[x/d\, 


where d is the greatest common divisor of m and n, and x is any real number. 
38. [M26] (E. Busche, 1909.) Prove that, for all real x and y with y > 0, 


E 

0<fc<y 



= l xy+ |z + ij(iyi -:y)J- 


In particular, when y is a positive integer n, we have the important formula 


1 

n — 1 

X H — 

+ •••+ £+ 

n. 

L n J 


39. [HM35] A function / for which f(x) + f(x +£) + ••• + f( x + 21 irO = f ( nx ) > 
whenever n is a positive integer, is called a replicative function. The previous exercise 
establishes the fact that is replicative. Show that the following functions are 
replicative: 

a) f(x) = x - 

b) f(x) = [x is an integer]; 

c ) f( x ) = is a positive integer]; 

d) f[x) = [there exists a rational number r and an integer m such that x = r7r + m]; 

e) three other functions like the one in (d), with r and/or m restricted to positive 
values; 

f) f[x) = log |2 sin7rx|, if the value f(x) = — oo is allowed; 

g) the sum of any two replicative functions; 

h) a constant multiple of a replicative function; 

i) the function g(x) = f[x — [ajJ ) , where f(x) is replicative. 
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40 . [HMJ,6] Study the class of replicative functions; determine all replicative functions 
of a special type. For example, is the function in (a) of exercise 39 the only continuous 
replicative function? It may be interesting to study also the more general class of 
functions for which 

n — 1 \ 

x H = a n J\nx) + b n . 

n ) 

Here a n and b n are numbers that depend on n but not on x. Derivatives and (if b n = 0) 
integrals of these functions are of the same type. If we require that b n = 0, we have, 
for example, the Bernoulli polynomials, the trigonometric functions cot7ra: and csc 2 7ra;, 
as well as Hurwitz’s generalized zeta function (‘(s, x) = Ylk > o !/(& + X Y f° r fixed s. 
With b n / 0 we have still other well-known functions, such as the psi function. 

41 . [M23] Let oi, d2, <13, ... be the sequence 1, 2, 2, 3, 3, 3, 4, 4, 4, 4, ... ; find an 
expression for a n in terms of n, using the floor and/or ceiling function. 

42 . [M24] (a) Prove that 

n n — 1 

y ak — na„ — y k(ak+i — a*), if n > 0. 
k= 1 fc=l 


f{x) + / 


( I+ s) 


+ ••• + / 


(b) The preceding formula is useful for evaluating certain sums involving the floor 
function. Prove that, if b is an integer > 2, 


ELlog^j = (n + 1) [log,, nj - (6*- logk n ^ +1 - b)/(b- 1). 
k - 1 

43. [ M23 } Evaluate 

44. [M 24 ] Show that Sfc>o L( n + jb k )/b k+1 \ = n, if b and n are integers, 

n > 0, and b > 2. What is the value of this sum when n < 0? 

► 45. [M28] The result of exercise 37 is somewhat surprising, since it implies that 


E 

0 <fc<n 


mk + x 

= v 

nk + x 

n 

Z—j 

0 <fe<m 

m 


This “reciprocity relationship” is one of many similar formulas (see Section 3.3.3). Show 
that for any function /, we have 


E / 

0<j <n 



0 <r<m 


In particular, prove that 


E 

0<j <n 


[mj/n\ + 1 
k 


+ E 

0<jf <m 


jn 

m 


( j \ _ ( m \ 

u-iJ n UJ- 


[Hint: Consider the change of variable r = [mj/nj. Binomial coefficients (^) are 
discussed in Section 1.2.6.] 

46 . [ M29 ] ( General reciprocity law.) Extend the formula of exercise 45 to obtain an 
expression for So <j«x n /( L m i/ n J)> where a is any positive real number. 


1.2.5 


PERMUTATIONS AND FACTORIALS 


45 


► 47. [MSI ] When p is an odd prime number, the Legendre symbol (|) is defined to 
be +1, 0, or —1, depending on whether q( p_1 U 2 modp is 1, 0, or p — 1. (Exercise 26 
proves that these are the only possible values.) 

a) Given that q is not a multiple of p, show that the numbers 

(_l)L2fcq/pj(2^.gmodp), 0 < k < p/2, 


are congruent in some order to the numbers 2, 4, . . . , p — 1 (modulo p). Hence 
(I) = (“l)" where ° = T,0<k<p/2l 2k( l/P\- 

b) Use the result of (a) to calculate (|). 

c) Given that q is odd, show that Y,o<k < P /2 l 2k< l/p\ = X)o<k<p /2 LWpJ (modulo 2). 
[Hint: Consider the quantity [(p — 1 — 2k)q/p\.] 

d) Use the general reciprocity formula of exercise 46 to obtain the law of quadratic 
reciprocity , (|) (|) = (-l) (p-1) ^ -1 ^ 4 , given that p and q are distinct odd primes. 

48. [M26] Prove or disprove the following identities, for integers m and n: 


(a) 


m + n — 1 
n 


m 

n 


n + 2 — [n/25j 


Sti -J- 24 

L 3 


L 25 J 


49. [M30] Suppose the integer-valued function f(x) satisfies the two simple laws 
(i) f(x + 1) = f(x) + 1; (ii) f(x) = f(f(nx)/n) for all positive integers n. Prove 
that either f(x) = [xj for all rational x, or f(x) = [a] for all rational x. 


1.2.5. Permutations and Factorials 

A permutation of n objects is an arrangement of n distinct objects in a row. 
There are six permutations of three objects { a,b,c }: 

abc, acb , bac , be a, cab , cba. (l) 

The properties of permutations are of great importance in the analysis of 
algorithms, and we will deduce many interesting facts about them later in this 
book.* Our first task is simply to count them: How many permutations of n 
objects are possible? There are n ways to choose the leftmost object, and once 
this choice has been made there are n — 1 ways to select a different object to place 
next to it; this gives us n{n - 1 ) choices for the first two positions. Similarly, we 
find that there are n - 2 choices for the third object distinct from the first two, 
and a total of n(n — 1 )(n — 2) possible ways to choose the first three objects. In 
general, if p n k denotes the number of ways to choose k objects out of n and to 
arrange them in a row, we see that 

Pnk = n(n - 1) ...(n- k+ 1). ( 2 ) 

The total number of permutations is therefore p nn — n(n — 1 ) ... ( 1 ). 

The process of constructing all permutations of n objects in an inductive 
manner, assuming that all permutations of n — 1 objects have been constructed, 

* In fact, permutations are so important, Vaughan Pratt has suggested calling them 
“perms.” As soon as Pratt’s convention is established, textbooks of computer science will 
be somewhat shorter (and perhaps less expensive). 
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is very important in our applications. Let us rewrite ( 1 ) using the numbers 
{1, 2, 3} instead of the letters {a, b , c}; the permutations are then 

1 2 3, 1 3 2, 2 1 3, 2 3 1, 3 1 2, 3 2 1. ( 3 ) 

Consider how to get from this array to the permutations of {1,2, 3, 4}. There 
are two principal ways to go from n — 1 objects to n objects. 

Method 1. For each permutation ai a 2 ... a n - 1 of {1, 2, . . . , n— 1}, form n 
others by inserting the number n in all possible places, obtaining 

110102 ... o n _i, a\ n a 2 ■ ■ ■ a n -i, ai «2 . . . na„_i, a\ 02 . . . a n -i n. 

For example, from the permutation 2 3 1 in ( 3 ), we get 423 1,243 1,234 1, 
2 3 14. It is clear that all permutations of n objects are obtained in this manner 
and that no permutation is obtained more than once. 

Method 2. For each permutation 0102 ••• o n -i of {1,2,..., n — 1} , form n 
others as follows: First construct the array 

d 2 . . . Qj n — \ 25 ^2 • • • ^n — 1 2 ’ * • • 5 ^ 1 ^2 • • • ®n — 1 2) ' 

Then rename the elements of each permutation using the numbers {1, 2, . . . , n}, 
preserving order. For example, from the permutation 2 3 1 in ( 3 ) we get 


231 

231 1, 

2 3 1 1 , 

231 1 

and, renaming, we get 




3421, 

3412, 

2413, 

2314. 


Another way to describe this process is to take the permutation aia 2 . . . a n -i 
and a number k, 1 < k < n; add one to each aj whose value is > k, thus 
obtaining a permutation bib? . . . 6 n _i of the elements {l,...,fc — 1 , fc + 1 , . . , , n}; 
then bib 2 ■ ■ ■ b n -ik is a permutation of { 1 , . . . , n }. 

Again it is clear that we obtain each permutation of n elements exactly once 
by this construction. Putting k at the left instead of the right, or putting k in 
any other fixed position, would obviously work just as well. 

If p n is the number of permutations of n objects, both of these methods show 
that p n = np n - 1 ; this offers us two further proofs that p n = n(n — 1 ) ... ( 1 ), as 
we already established in Eq. ( 2 ). 

The important quantity p n is called n factorial and it is written 

n 

n! = l- 2 - ...-n = k. ( 4 ) 

fc= 1 

Our convention for vacuous products (Section 1.2.3) gives us the value 

0 ! = 1 , ( 5 ) 

and with this convention the basic identity 

n! = (n — 1)! n 

is valid for all positive integers n. 


( 6 ) 
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Factorials come up sufficiently often in computer work that the reader is 
advised to memorize the values of the first few: 


0! = 1, 1! = 1, 2! = 2, 3! = 6, 4! = 24, 5! = 120. 


The factorials increase very rapidly; for example, 1000! is an integer with over 
2500 decimal digits. 

It is helpful to keep the value 10! = 3,628,800 in mind; one should remember 
that 10! is about 3| million. In a sense, this number represents an approximate 
dividing line between things that are practical to compute and things that are 
not. If an algorithm requires the testing of more than 10! cases, it may consume 
too much computer time to be practical. On the other hand, if we decide to 
test 10! cases and each case requires, say, one millisecond of computer time, 
then the entire run will take about an hour. These comments are very vague, of 
course, but they can be useful to give an intuitive idea of what is computationally 
feasible. 

It is only natural to wonder what relation n\ bears to other quantities in 
mathematics. Is there any way to tell how large 1000! is, without laboriously 
carrying out the multiplications implied in Eq. (4)? The answer was found by 
James Stirling in his famous work Methodus Differentialis (1730), page 137; we 
have 



The sign that appears here denotes “approximately equal,” and “e” is the 
base of natural logarithms introduced in Section 1.2.2. We will prove Stirling’s 
approximation (7) in Section 1.2.11.2. Exercise 24 gives a simple proof of a less 
precise result. 

As an example of the use of this formula, we may compute 

40320 = 8! w Ay/n ( - J = 2 26 ^e“ 8 « 67108864 • 1.77245 ■ 0.00033546 « 39902. 

In this case the error is about 1%; we will see later that the relative error is 
approximately l/(12n). 

In addition to the approximate value given by Eq. (7), we can also rather 
easily obtain the exact value of n! factored into primes. In fact, the prime p is a 
divisor of nl with the multiplicity 


M = 

n 

+ 

n 

9 

+ 

n 


-V- 


ip J 


ip^ J 


= E 


k> 0 


(8) 


For example, if n = 1000 and p = 3, we have 


h = 


1000 
. 3 . 


1000 
. 9 . 


1000 
. 27 . 


1000 
_ 81 . 


1000 
. 243 . 


+ 


1000 
. 729 . 


= 333 + 111 + 37 + 12 + 4 + 1 = 498, 


so 1000! is divisible by 3 498 but not by 3 499 . Although formula (8) is written as an 
infinite sum, it is really finite for any particular values of n and p, because all of 
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the terms are eventually zero. It follows from exercise 1.2.4-35 that [n/p k+1 J = 
[|n/p fe J/pJ; this fact facilitates the calculation in Eq. (8), since we can just 
divide the value of the previous term by p and discard the remainder. 

Equation (8) follows from the fact that \n/p k J is the number of integers 
among {1,2, ...,n} that are multiples of p k . If we study the integers in the 
product (4), any integer that is divisible by p j but not by p j+1 is counted exactly 
j times: once in [n/pj, once in [n/p 2 J, . . . , once in \n/pi\. This accounts for 
all occurrences of p as a factor of n\. [See A. M. Legendre, Ess ai sur la Theorie 
des Nombres, second edition (Paris: 1808), page 8.] 

Another natural question arises: Now that we have defined n! for non- 
negative integers n, perhaps the factorial function is meaningful also for rational 
values of n, and even for real values. What is ({)!. for example? Let us illustrate 
this point by introducing the “termial” function 

n 

71? = 1 + 2-1 — + n = 'y k, (9) 

k= 1 

which is analogous to the factorial function except that we are adding instead 
of multiplying. We already know the sum of this arithmetic progression from 
Eq. 1.2.3 -(i 5 ): 

7i? = \n{n + 1). (10) 

This suggests a good way to generalize the “termial” function to arbitrary n, 
by using (10) instead of (9). We have (|)? = |. 

Stirling himself made several attempts to generalize n! to noninteger n. He 
extended the approximation (7) into an infinite sum, but unfortunately the sum 
did not converge for any value of n; his method gave extremely good approxi- 
mations, but it couldn’t be extended to give an exact value. [For a discussion 
of this somewhat unusual situation, see K. Knopp, Theory and Application of 
Infinite Series, 2nd ed. (Glasgow: Blackie, 1951), 518-520, 527, 534.] 

Stirling tried again, by noticing that 

n! = 1 +( 1 -^)n+ (l- ^ + ^)n(n-l) 

+ (1 - ^ - ^) n ( n ~ + •••• ( xl ) 

(We will prove this formula in the next section.) The apparently infinite sum 
in Eq. (11) is in reality finite for any nonnegative integer n; however, it does 
not provide the desired generalization of 71!, since the infinite sum does not exist 
except when 71 is a nonnegative integer. (See exercise 16.) 

Still undaunted, Stirling found a sequence a\, a ^, . . . such that 

Inn! = cqn + a 2 n(n - 1) 4 = ^ a/c+i ( n-j ). (12) 

fc> 0 0<j<k 

He was unable to prove that this sum defined n! for all fractional values of n, 
although he was able to deduce the value of (|)! = y/ir/2. 
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At about the same time, Leonhard Euler considered the same problem, and 
he was the first to find the appropriate generalization: 


n! 


i i ; 1 1 

m—too 


m"m! 

(n + l)(n + 2 ) . . . (n + to) 


( 13 ) 


Euler communicated this idea in a letter to Christian Goldbach on October 13, 
1729. His formula defines n! for any value of n except negative integers (when 
the denominator becomes zero); in such cases n! is taken to be infinite. Exercises 
8 and 22 explain why Eq. ( 13 ) is a reasonable definition. 

Nearly two centuries later, in 1900, C. Hermite proved that Stirling’s idea 
( 12 ) actually does define n! successfully for nonintegers n, and that in fact Euler’s 
and Stirling’s generalizations are identical. 

Many notations were used for factorials in the early days. Euler actually 
wrote [n], Gauss wrote nn, and the symbols \n_ and nj were popular in England 
and Italy. The notation n!, which is universally used today when n is an inte- 
ger, was introduced by a comparatively little known mathematician, Christian 
Kramp, in an algebra text [Elemens d’Arithmetique Universelle (Cologne: 1808), 
page 219]. 

When n is not an integer, however, the notation n! is less common; instead 
we customarily employ a notation due to A. M. Legendre: 


n! = T(n + 1 ) = nT(n). 


(m) 


This function T(x) is called the gamma function, and by Eq. ( 13 ) we have the 
definition 


T( X ) 


x\ m to! 

— = lim — 77 — r. 

X m->oo j 1 + 1) a: + 2 ... X + T71 


(!5) 


A graph of T(a:) is shown in Fig. 7. 

Equations ( 13 ) and ( 15 ) define factorials and the gamma function for com- 
plex values as well as real values; but we generally use the letter 2 , instead of n 
or x, when thinking of a variable that has both real and imaginary parts. The 
factorial and gamma functions are related not only by the rule z\ = T(z + 1 ) but 
also by 


(-*)!T(z) 


7T 

sin 7 rz ’ 


(16) 


which holds whenever 2 is not an integer. (See exercise 23.) 

Although F(z) is infinite when 2 is zero or a negative integer, the function 
1 /T(z) is well defined for all complex 2 . (See exercise 1.2.7-24.) Advanced 
applications of the gamma function often make use of an important contour 
integral formula due to Hermann Hankel: 


1 1 f e* dt 

T( 2 ) 27ri J t z 


(m) 


the path of complex integration starts at — 00 , then circles the origin in a 
counterclockwise direction and returns to — 00 . [ Zeitschrift fur Math, und Physik 
9 (1864), 1-21.] 
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- 4 - 3 - 2-10 1 2 3 4 


Fig. 7. The function T(a;) = (x — 1)!. The local minimum at X has the coordinates 
(1.46163 21449 68362 34126 26595, 0.88560 31944 10888 70027 88159). 

Many formulas of discrete mathematics involve factorial-like products known 
as factorial powers. The quantities x- and x k (read, “x to the k falling” and “x 
to the k rising” ) are defined as follows, when A; is a positive integer: 


k — 1 


x- — x(x - 1) . . . (x - k + 1) = (x - j)-, 

1=0 

fe-t 

(18) 

x k — x(x + 1) . . . (a: + k - 1) = JJ(z + j). 

(19) 


3 = 0 

Thus, for example, the number p n k of ( 2 ) is just n-. Notice that we have 


x k = (x + k — 1)- = (— l) fc (— a:)-. 

( 20 ) 

The general formulas 


r k x\ fc I> + fc) 

(■ x — k)\ ’ T(a;) 

( 21 ) 


can be used to define factorial powers for other values of k. [The notations x k and 
x- are due respectively to A. Capelli, Giornale di Mat. di Battaglini 31 (1893), 
291-313, and L. Toscano, Comment. Accademia della Scienze 3 (1939), 721-757.] 
The interesting history of factorials from the time of Stirling to the present 
day is traced in an article by P. J. Davis, “Leonhard Euler’s integral: A historical 
profile of the gamma function,” AMM 66 (1959), 849-869. See also J. Dutka, 
Archive for History of Exact Sciences 31 (1984), 15-34. 
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EXERCISES 

1 . [00] How many ways are there to shuffle a 52-card deck? 

2 . [10] In the notation of Eq. ( 2 ), show that p n ( n - 1 ) = Pnn, and explain why this 
happens. 

3 . [10] What permutations of {1, 2, 3,4, 5} would be constructed from the permuta- 
tion 3 12 4 using Methods 1 and 2, respectively? 

► 4 . [13] Given the fact that log 10 1000! = 2567.60464..., determine exactly how 
many decimal digits are present in the number 1000!. What is the most significant 
digit? What is the least significant digit? 

5 . [15] Estimate 8 ! using the following more exact version of Stirling’s approxi- 
mation: 



► 6. [17] Using Eq. ( 8 ), write 20! as a product of prime factors. 

7. [M10] Show that the “generalized termial” function in Eq. ( 10 ) satisfies the 
identity x? — x + (x — 1 )? for all real numbers x. 

8 . [HM15] Show that the limit in Eq. ( 13 ) does equal n\ when n is a nonnegative 
integer. 

9. [M10] Determine the values of E(|) and T(— |), given that (|)! = sfi r/2. 

► 10 . [HM20] Does the identity r(x + 1) = xV(x) hold for all real numbers x? (See 
exercise 7.) 

11 . [ M15 ] Let the representation of n in the binary system be n = 2 ei +2 62 + • • - + 2 er , 
where ei > e 2 > • ■ • > e r > 0. Show that n! is divisible by 2 n-r but not by 2 "~ r+1 . 

► 12 . [M22] (A. Legendre, 1808.) Generalizing the result of the previous exercise, let 
p be a prime number, and let the representation of n in the p-ary number system be 
n = akp k + ak-ip k ~ 1 + • • • + aip + 00 . Express the number p of Eq. ( 8 ) in a simple 
formula involving n, p, and a’s. 

13 . [M23] ( Wilson’s theorem, actually due to Leibniz, 1682.) If p is prime, then 
(p — 1)! mod p = p — 1. Prove this, by pairing off numbers among {1, 2, . . . , p — 1} 
whose product modulo p is 1 . 

► 14 . [M28] (L. Stickelberger, 1890.) In the notation of exercise 12, we can determine 
n\ modp in terms of the p-ary representation, for any positive integer n, thus general- 
izing Wilson’s theorem. In fact, prove that n!/p M = (— l) M ao! ai! . . . a*,! (modulo p). 

15 . [HM15] The permanent of a square matrix is defined by the same expansion as 
the determinant except that each term of the permanent is given a plus sign while the 
determinant alternates between plus and minus. Thus the permanent of 


( a b c\ 
d e / 
g h i ) 

is aei + bfg + cdh + gee + hfa + idb. What is the permanent of 


lxl 

1x2 .. 

1 x n 

2x1 

2x2 

2 X n 

n x 1 

n x 2 . , 

n X n 
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16 . [HM15] Show that the infinite sum in Eq. (n) does not converge unless n is a 
nonnegative integer. 

17 . [HM20] Prove that the infinite product 

tt (n + cti) ... (n + a k ) 

4i (n + Pi) ■■■ (n + Pk) 

equals T(1 + /3i) ... r(l + /3 k )/T{l + on) ... r(l + a fc ), if qi + \- a k = /?i + h j3 k 

and if none of the /3’s is a negative integer. 

18 . [M20] Assume that n/2 = (This is “Wallis’s product,” 

obtained by J. Wallis in 1655, and we will prove it in exercise 1.2.6-43.) Using the 
previous exercise, prove that (|)! = \fn/2. 

19 . [HM22] Denote the quantity appearing after “linim-KJo” in Eq. (15) by P m (x). 
Show that 

f' rn / f \m r 1 

T m {x) = J [l- -) t x ~ 1 dt = m x J (l-trf-'dt, if x > 0. 

20 . [HM21] Using the fact that 0 < e~ 4 - (1 — t/m) rn < f 2 e _t /m, if 0 < t < m, and 
the previous exercise, show that T(x) = J 0 °° e -4 ** -1 dt, if x > 0. 

21 . [HM25] (L. F. A. Arbogast, 1800.) Let D^u represent the fcth derivative of a 
function u with respect to x. The chain rule states that D].w = D' u w D).u. If we apply 
this to second derivatives, we find D\ w = Dlw(Dlu) 2 + D l u wDlu. Show that the 
general formula is 


D~w ■ 


E E 


3=0 


ki+k 2 -\ \-k n =j 

ki +2k 2 -\ \-nk n — 

k\ ,k 2 ,...,k n >0 




► 22 . [HM20] Try to put yourself in Euler’s place, looking for a way to generalize n\ 
to noninteger values of n. Since (n + §)!/«! times ((n + |) + |)!/(n + |)! equals 
(n + l)!/n! = n + 1 , it seems natural that (n + |)!/n! should be approximately y/n. 
Similarly, (n+ \ )\/n\ should be « yfn. Invent a hypothesis about the ratio ( n-\-x)\/n\ 
as n approaches infinity. Is your hypothesis correct when x is an integer? Does it tell 
anything about the appropriate value of x\ when x is not an integer? 

23 . [HM20] Prove (16), given that 7rz n~=i(! - z 2 /n 2 ) = sinTrz. 

► 24 . [HM21] Prove the handy inequalities 


n" ^ ^ n n+1 

< n! < ? integer n > 1. 

[Hint: 1 + x < e x for all real x; hence ( k + l)/k < e 1 ^ < k/(k — 1 ).] 

25. [M20] Do factorial powers satisfy a law analogous to the ordinary law of expo- 
nents, x m+n = x m x n ? 


1.2.6. Binomial Coefficients 

The combinations of n objects taken k at a time are the possible choices of 
k different elements from a collection of n objects, disregarding order. The 
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combinations of the five objects {a, b , c, d, e} taken three at a time are 

abc, abd, abe, acd, ace , ade , bed , 6ce, bde , cde. ( 1 ) 


It is a simple matter to count the total number of fc-combinations of n objects: 
Equation ( 2 ) of the previous section told us that there are n(n — 1) . . . (n — k + 1) 
ways to choose the first k objects for a permutation; and every fc-combination 
appears exactly k\ times in these arrangements, since each combination appears 
in all its permutations. Therefore the number of combinations, which we denote 
by (fc), is 

( n\ n(n — 1) . . . [n — k + 1) . . 

IJ = Jfe(Jb- !)...(!) ' W 


For example, 



5-4-3 

3-2-1 


10 , 


which is the number of combinations we found in ( 1 ). 

The quantity (£), read “n choose k,” is called a binomial coefficient ; these 
numbers have an extraordinary number of applications. They are probably the 
most important quantities entering into the analysis of algorithms, so the reader 
is urged to become familiar with them. 

Equation ( 2 ) may be used to define (£) even when n is not an integer. To 
be precise, we define the symbol (£) for all real numbers r and all integers k as 
follows: 


r\ r(r — 1) . . . (r — k + 1) 
k) = k(k- 1)...(1) 


k\ 


n 


r+l-j 

j 


integer k > 0; 



In particular cases we have 



integer k < 0. 


G)=^ 


(3) 


(4) 


Table 1 gives values of the binomial coefficients for small integer values of r 
and k ; the values for 0 < r < 4 should be memorized. 

Binomial coefficients have a long and interesting history. Table 1 is called 
“Pascal’s triangle” because it appeared in Blaise Pascal’s Traite du Triangle 
Arithmetique in 1653. This treatise was significant because it was one of the 
first works on probability theory, but Pascal did not invent the binomial co- 
efficients (which were well-known in Europe at that time). Table 1 also ap- 
peared in the treatise Szu-yiian Yii-chien (“The Precious Mirror of the Four 
Elements”) by the Chinese mathematician Chu Shih-Chieh in 1303, where they 
were said to be an old invention. Yang Hui, in 1261, credited them to Chia 
Hsien (c. 1100), whose work is now lost. The earliest known detailed discussion 
of binomial coefficients is in a tenth-century commentary, due to Halayudha, on 
an ancient Hindu classic, Pingala’s Chandahsastra. [See G. Chakravarti, Bull. 


54 


BASIC CONCEPTS 


Table 1 

TABLE OF BINOMIAL COEFFICIENTS (PASCAL’S TRIANGLE) 


1.2.6 


r 

n n n n n n n n n n 

\0j VI/ V2 ) \3 ) \4/ V 5 / \6j \7J \sJ W 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

2 

1 

2 

1 

0 

0 

0 

0 

0 

0 

0 

3 

1 

3 

3 

1 

0 

0 

0 

0 

0 

0 

4 

1 

4 

6 

4 

1 

0 

0 

0 

0 

0 

5 

1 

5 

10 

10 

5 

1 

0 

0 

0 

0 

6 

1 

6 

15 

20 

15 

6 

1 

0 

0 

0 

7 

1 

7 

21 

35 

35 

21 

7 

1 

0 

0 

8 

1 

8 

28 

56 

70 

56 

28 

8 

1 

0 

9 

1 

9 

36 

84 

126 

126 

84 

36 

9 

1 


Calcutta Math. Soc. 24 (1932), 79-88.] Another 
Indian mathematician, Mahavlra, had previously 
explained rule ( 3 ) for computing (£) in Chapter 6 
of his Ganita Sara Sangraha, written about 850; 
and in 1150 Bhaskara repeated Mahavlra’s rule 
near the end of his famous book Lilavati. For small 
values of k, binomial coefficients were known much 
earlier; they appeared in Greek and Roman writ- 
ings with a geometric interpretation (see Fig. 8 ). 

The notation (f) was introduced by Andreas von 
Ettingshausen in §31 of his book Die combina- 
torische Analysis (Vienna: 1826). 

The reader has probably noticed several in- 
teresting patterns in Table 1 . Binomial coeffi- 
cients satisfy literally thousands of identities, and 
for centuries their amazing properties have been 
continually explored. In fact, there are so many 
relations present that when someone finds a new 
identity, not many people get excited about it any 
more, except the discoverer. In order to manipu- 
late the formulas that arise in the analysis of algorithms, a facility for handling 
binomial coefficients is a must, and so an attempt has been made in this section 
to explain in a simple way how to maneuver with these numbers. Mark Twain 
once tried to reduce all jokes to a dozen or so primitive kinds (farmer’s daughter, 
mother-in-law, etc.); we will try to condense the thousands of identities into a 
small set of basic operations with which we can solve nearly every problem 
involving binomial coefficients that we will meet. 

In most applications, both of the numbers r and k that appear in (£) will 
be integers, and some of the techniques we will describe are applicable only in 
such cases. Therefore we will be careful to list, at the right of each numbered 



Fig. 8. Geometric interpre- 
tation of ("3 2 ) , n = 4. 
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equation, any restrictions on the variables that appear. For example, Eq. (3) 
mentions the requirement that k is an integer; there is no restriction on r. The 
identities with fewest restrictions are the most useful. 

Now let us study the basic techniques for operating on binomial coefficients: 


A. Representation by factorials. From Eq. (3) we have immediately 



n! 

k\ ( n — k)\ ’ 


integer n > integer k > 0. 


( 5 ) 


This allows combinations of factorials to be represented as binomial coefficients 
and conversely. 


B. Symmetry condition. From Eqs. (3) and (5), we have 

(^j — ^ n k), integer n > 0, integer k. (6) 

This formula holds for all integers k. When k is negative or greater than n, the 
binomial coefficient is zero (provided that n is a nonnegative integer). 

C. Moving in and out of parentheses. From the definition (3), we have 

(D = Kfc-D’ integer * 54 °- w 

This formula is very useful for combining a binomial coefficient with other parts 
of an expression. By elementary transformation we have the rules 

Hina: 1 .)- 

the first of which is valid for all integers k, and the second when no division by 
zero has been performed. We also have a similar relation: 

inte6er * # r ' (8) 

Let us illustrate these transformations, by proving Eq. (8) using Eqs. (6) 
and (7) alternately: 

( r W r r ~ 1 U-i-f- 1 ). 

\kJ \r — kj r — k\r — \ — kJ r — k\ k J 

[Note: This derivation is valid only when r is a positive integer 7^ k, because of 
the constraints involved in Eqs. (6) and (7); yet Eq. (8) claims to be valid for 
arbitrary r k. This can be proved in a simple and important manner: We 
have verified that 

r C* I )= (r - k, (D 

for infinitely many values of r. Both sides of this equation are polynomials in r. 
A nonzero polynomial of degree n can have at most n distinct zeros; so (by 
subtraction) if two polynomials of degree < n agree at n + 1 or more different 
points, the polynomials are identically equal. This principle may be used to 
extend the validity of many identities from integers to all real numbers.] 
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D. Addition formula. The basic relation 

(I) = (V) + G-D’ integer/c ’ (9) 

is clearly valid in Table 1 (every value is the sum of the two values above and 
to the left) and we may easily verify it in general from Eq. (3). Alternatively, 
Eqs. (7) and (8) tell us that 

GCVGCH-O-GHQ 

Equation (9) is often useful in obtaining proofs by induction on r, when r is an 
integer. 


E. Summation formulas. Repeated application of (9) gives 

GMCG + GCMCG+G-iMCD 


or 


GMlCHTMClMCiC 


r - 2 
k 


Thus we are led to two important summation formulas that can be expressed as 
follows: 




\ fr + n\ /r + n + 1\ 

J + "' + l n )-{ n )’ 
integer n > 0. 



integer to > 0, integer n > 0. 


(10) 


(11) 


Equation (11) can easily be proved by induction on n, but it is interesting 
to see how it can also be derived from Eq. (10) with two applications of Eq. (6): 


eo - £ rr)- e ( 


(m + k 


0 <k<n 


0<m+k<n 


—m<k< 0 


e rn 


0 </e<n— m 


= 0 


m + (n — m) + 1 
n — m 


n+ 1 
to + 1 


assuming that n > m. If n < m, Eq. (11) is obvious. 

Equation (11) occurs very frequently in applications; in fact, we have already 
derived special cases of it in previous sections. For example, when m = 1 , we 
have our old friend, the sum of an arithmetic progression: 



0 + 1 + 1 -n = 



(n + l)n 


2 
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can 


Suppose that we want a simple formula for the sum l 2 + 2 2 H In 2 . This 

be obtained by observing that k 2 = 2 + (*) ; hence 


xexxM’Tmt)- 

fc=0 k = 0 v ' 


And this answer, obtained in terms of binomial coefficients, can be put back into 
polynomial notation if desired: 


l 2 + 2 2 H 1- n 2 


o (n + l)n(n — 1) 

6 


(n + l)n 
2 


\n{n + \)(n + 1). (12) 


The sum l 3 + 2 3 -1 1-n 3 can be obtained in a similar way; any polynomial 

a 0 + aik + a%k 2 + b a m k m can be expressed as b 0 (fj) + 61 (J) + b b m ( m ) 

for suitably chosen coefficients bo, , b m . We wifi return to this subject later. 


F. The binomial theorem. Of course, the binomial theorem is one of our 
principal tools: 

C x + y) r = y^ J (j^)x k y r ~ k , integer r > 0. (13) 


For example, ( x + y ) 4 = x 4 + 4 x 3 y + &x 2 y 2 + 4 xy 3 + y 4 . (At last we are able to 
justify the name “binomial coefficient” for the numbers (£) .) 

ft is important to notice that we have written * n Eq. (13), rather than 
Y^[=o as m ight have been expected. If no restriction is placed on k, we are 
summing over all integers, —00 < k < +00; but the two notations are exactly 
equivalent in this case, since the terms in Eq. (13) are zero when fc < 0 or 
k > r. The simpler form is to be preferred, since all manipulations with 
sums are simpler when the conditions of summation are simpler. We save a 
good deal of tedious effort if we do not need to keep track of the lower and/ or 
upper limits of summation, so the limits should be left unspecified whenever 
possible. Our notation has another advantage also: If r is not a nonnegative 
integer, Eq. (13) becomes an infinite sum, and the binomial theorem of calculus 
states that Eq. (13) is valid for all r, if \x/y\ < 1. 

It should be noted that formula (13) gives 

0° = 1. (14) 


We will use this convention consistently. 

The special case y = 1 in Eq. (13) is so important, we state it specially: 


E(lH = < 1+ *> r 

k 


integer r > 0 or |rc| < 1. 


(15) 


The discovery of the binomial theorem was announced by Isaac Newton in 
letters to Oldenburg on June 13, 1676 and October 24, 1676. [See D. Struik, 
Source Book in Mathematics (Harvard Univ. Press, 1969), 284-291.] But he 
apparently had no real proof of the formula; at that time the necessity for rigorous 
proof was not fully realized. The first attempted proof was given by L. Euler 
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in 1774, although his effort was incomplete. Finally, C. F. Gauss gave the first 
actual proof in 1812. In fact, Gauss’s work represented the first time anything 
about infinite sums was proved satisfactorily. 

Early in the nineteenth century, N. H. Abel found a surprising generalization 
of the binomial formula (13): 

{x + y) n = J2(J^jx(x-kz) k ~ 1 (y + kz) n ~ k , integer n > 0, x / 0. (16) 

k 

This is an identity in three variables, x, y, and 2 (see exercises 50 through 52). 
Abel published and proved this formula in Volume 1 of A. L. Crelle’s soon- 
to-be-famous Journal fur die reine und angewandte Mathematik (1826), pages 
159-160. It is interesting to note that Abel contributed many other papers to the 
same Volume 1, including his famous memoirs on the unsolvability of algebraic 
equations of degree 5 or more by radicals, and on the binomial theorem. See 
H. W. Gould, AMM 69 (1962), 572, for a number of references to Eq. (16). 

G. Negating the upper index. The basic identity 

(fc) = (“ 1 ) fe ( fc l integer k, (i 7 ) 

follows immediately from the definition (3) when each term of the numerator is 
negated. This is often a useful transformation on the upper index. 

One easy consequence of Eq. (17) is the summation formula 

£( fc ) (_1) * = (°) ~ (l) + +(_ 1 ) "(n) =:(_1)n ( 7 'n 1 )’ inte S erra - 

(18) 

This identity could be proved by induction using Eq. (9), but we can use Eqs. (17) 
and (10) directly: 


SO'-'-s (‘T‘)-rr)-<->-( 


r — 1 


Another important application of Eq. (17) can be made when r is an integer: 

(m) = ( I” ’ int6ger n - °> inte S er m ■ (!9) 

(Set r — n and k — n — m in Eq. (17) and use (6).) We have moved n from the 
upper position to the lower. 


H. Simplifying products. When products of binomial coefficients appear, they 
can usually be reexpressed in several different ways by expanding into factorials 
and out again using Eq. (5). For example, 


/ rWm\ _ ( r \ ( r — k \ 
\mj V k ) \k / \m — k J ' 


( 20 ) 


integer m, integer k. 
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It suffices to prove Eq. (20) when r is an integer > m (see the remarks after 
Eq. (8)) , and when 0 < k < m. Then 

/ r \ /m\ __ r!m! H.(r-fc)! = /r\ / r-k \ 

VmJvfc/ rn\ (r — m)! k\ (m — fc)! k\(r — k)\(m—k)\(r—m)\ \k)\m — k ) 

Equation (20) is very useful when an index (namely to) appears in both the 
upper and the lower position, and we wish to have it appear in one place rather 
than two. Notice that Eq. (7) is the special case of Eq. (20) when k = 1. 


I. Sums of products. To complete our set of binomial-coefficient manipu- 
lations, we present the following very general identities, which are proved in 
the exercises at the end of this section. These formulas show how to sum over a 
product of two binomial coefficients, considering various places where the running 
variable k might appear: 


£(!)(„’*) = ( r »’)' inttgein - 
k 

( r \( s \ / r + s \ 

^\m + kj \n + kJ \r-m + nJ ’ 


(21) 


r + s 

r — m + n / 

integer m, integer n, integer r > 0. (22) 


^2 (Jj^j ^ ^ (-l) r fc= ( S )’ int e g er integer r > 


0 . 


( 2 3) 


k = 0 


integer t > 0, integer r > 0, integer m > 0. (24) 


A /r — k\ ( s + k\ _ /r + s + 1 \ 
m Jv n ) Vm + n + 1/’ 

k=0 


integer n > integer s > 0, integer m > 0, integer r > 0. (25) 


% . / v tk ' 
k 

k> 0 


K s — t(n — k)\ r _/r + s-tn\ 
n - k ) r — tk V n )' 


integer n. (26) 


Of these identities, Eq. (21) is by far the most important, and it should be 
memorized. One way to remember it is to interpret the right-hand side as the 
number of ways to select n people from among r men and s women; each 
term on the left is the number of ways to choose k of the men and n — k 
of the women. Equation (21) is commonly called Vandermonde’s convolution, 
since A. Vandermonde published it in Mem. Acad. Roy. Sciences (Paris, 1772), 
part 1, 489-498. However, it had appeared already in Chu Shih-Chieh’s 1303 
treatise mentioned earlier [see J. Needham, Science and Civilisation in China 3 
(Cambridge University Press, 1959), 138-139]. 
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If r = tk in Eq. (26), we avoid the zero denominator by canceling with 
a factor in the numerator; therefore Eq. (26) is a polynomial identity in the 
variables r, s, t. Obviously Eq. (21) is a special case of Eq. (26) with t — 0. 

We should point out a nonobvious use of Eqs. (23) and (25): It is often 
helpful to replace the simple binomial coefficient on the right-hand side by the 
more complicated expression on the left, interchange the order of summation, 
and simplify. We may regard the left-hand sides as expansions of 

f ' ) in terms of ( S + k ). 

Formula (23) is used for negative a, formula (25) for positive a. 

This completes our study of binomial-coefficientology. The reader is advised 
to learn especially Eqs. (5), (6), (7), (9), (13), (17), (20), and (21) — frame them 
with your favorite highlighter pen! 

With all these methods at our disposal, we should be able to solve almost 
any problem that comes along, in at least three different ways. The following 
examples illustrate the techniques. 

Example 1. When r is a positive integer, what is the value of JZQ j (*)jfe? 
Solution. Formula (7) is useful for disposing of the outside k: k 

sOO-sQaiD—EOC:;). 

K k k 

Now formula (22) applies, with m = 0 and n = - 1 . The answer is therefore 


inteser 


r > 0. 


Example 2. What is the value of T.(n + k 
integer? 


VV 2 k 

k 


k + 1 


, if n is a nonnegative 


Solution. This problem is tougher; the summation index k appears in six places! 
First we apply Eq. (20), and we obtain 


ST'(n + k\ /n\ (~l) fc 

■“ V k J \ k J k + 1 

k 

We can now breathe more easily, since several of the menacing characteristics 
of the original formula have disappeared. The next step should be obvious; we 
apply Eq. (7) in a manner similar to the technique used in Example 1: 


v-' fn-\- k \ (n -\- 1 




-)( 


k + 1 


(-l) fc 


(27) 


Good, another k has vanished. At this point there are two equally promising 
lines of attack. We can replace the (”+ fc ) by ( n + fc ) , assuming that k > 0 , and 
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fc> 0 


+ 1 \(- 1 )* 


k + 1 ) n + 1 


all 


n — 1 + k 
n 

n — 1 + k 
n 


)( 

)( 


n + 1 


(-1)* 


n+l v V — 1 / n + 1 V n ) n + lV n ) 

The binomial coefficient ( n ~ 1 ) equals zero except when n = 0, in which case it 
equals one. So we can conveniently state the answer to our problem as [n = 0], 
using Iverson’s convention (Eq. 1.2.3-(i6)), or as S n 0, using the Kronecker delta 
(Eq. 1.2.3 -(i 9 )). 

Another way to proceed from Eq. (27) is to use Eq. (17), obtaining 


E 

k 


-{n + 1 ) 
k 


/n+l\ 1 

Vfc + lJn+1 


We can now apply Eq. (22), which yields the sum 
'n+ 1 - {n + 1 ) 


0 


n + 1 — 1 + 0 
Once again we have derived the answer 

'n + k\ ( 2 k\ (— l) fc 
k + 1 


n + 1 Vn/n + 1 


v / 7i -|“ k \ / *2k \ 

^ V 2k )\k ) 


= 6 . 


n0 5 


integer n > 0. 


28) 


E / n -\- k \ { 2 k \ ( — 1) 

( , ) ( ) — — — , for positive inte- 

i \ tyi I 2 k J \ k J k I 1 

gers m and n! k 

Solution. If m were zero, we would have the same formula to work with that 
we had in Example 2. However, the presence of m means that we cannot even 
begin to use the method of the previous solution, since the first step there was 
to use Eq. (20) — which no longer applies. In this situation it pays to complicate 
things even more by replacing the unwanted k ) ^ a sum terms °f tire 
form ( x ^ k ) , since our problem will then become a sum of problems that we know 
how to solve. Accordingly, we use Eq. (25) with 


and we have 


r — n + k — 1, m = 2k, s — 0, n = m — 1, 

i-if 


E E ( 

k 0<j<n+k—l 


n + k — 1 
2k 




k + 1 


( 2 9 ) 
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We wish to perform the summation on k first; but interchanging the order of 
summation demands that we sum on the values of k that are > 0 and > j -n + 1. 
Unfortunately, the latter condition raises problems, because we do not know the 
desired sum if j > n. Let us save the situation, however, by observing the terms 
of (29) are zero when n < j < n + k - 1 . This condition implies that k > 1; thus 
0 < n + k — 1— j < k — 1< 2 k, and the first binomial coefficient in (29) will 
vanish. We may therefore replace the condition on the second sum by 0 < j < n, 
and the interchange of summation is routine. Summing on k by Eq. (28) now 
gives 

^ (m- i) 5 ( »-i-i )0 ' 

0<j<n 

and all terms are zero except when j = n — 1 . Hence our final answer is 

p-M. 

\m — 1/ 

The solution to this problem was fairly complicated, but not really myste- 
rious; there was a good reason for each step. The derivation should be studied 
closely because it illustrates some delicate maneuvering with the conditions in 
our equations. There is actually a better way to attack this problem, however; 
it is left to the reader to figure out a way to transform the given sum so that 
Eq. (26) applies (see exercise 30 ). 

Example 4 . Prove that 

Y Ak(r, t)A n -k{s, t) = A n (r + s, t), integer n > 0, (30) 

fc 

where A n (x,t) is the nth degree polynomial in x that satisfies 

. , . (x — nt\ x 

A n {x,t) = ( -, for x / nt. 

\ n 1 x — nt 

Solution. We may assume that r ^ kt 7^ s for 0 < k < n, since both sides of 
(30) are polynomials in r, s, t. Our problem is to evaluate 

/ r — kt\ f s — (n — k)t\ r s 

■— V k ) \ n — k J r — kt s — (n — k)t ’ 

which, if anything, looks much worse than our previous horrible problems! Notice 
the strong similarity to Eq. (26), however, and also note the case t = 0 . 

We are tempted to change 

fr — kt\ r /r — kt - l\r 

V k ) r — kt t0 \ k — 1 ) k' 

except that the latter tends to lose the analogy with Eq. (26) and it fails when 
k = 0 . A better way to proceed is to use the technique of partial fractions, 
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whereby a fraction with a complicated denominator can often be replaced by a 
sum of fractions with simpler denominators. Indeed, we have 

1 1 _ 1 

r — kt s — (n — k)t r + s — nt 
Putting this into our sum we get 

s / r — kt\ / s — (n — k)t\ 

r + s — nt V k J \ n — k ) r — kt 

k 

r ^ (r — kt\ ( s — [n — k)t\ s 

r + s - nt “V k /\ n — k ) s — (n — k)t’ 

and Eq. (26) evaluates both of these formulas if we change k to n — k in the 

second; the desired result follows immediately. Identities (26) and (30) are due 
to H. A. Rothe, Formulas de Serierum Reversione (Leipzig: 1793); special cases of 
these formulas are still being “discovered” frequently. For the interesting history 
of these identities and some generalizations, see H. W. Gould and J. Kaucky, 
Journal of Combinatorial Theory 1 (1966), 233-247. 

Example 5. Determine the values of do, a±, a 2 , ... such that 

n\ — ao + a\ n + a^n^n — 1) + azn(n — l)(n — 2) + • ■ • (31) 

for all nonnegative integers n. 

Solution. Equation 1.2.5-(n), which was presented without proof in the previous 
section, gives the answer. Let us pretend that we don’t know it yet. It is clear 
that the problem does have a solution, since we can set n = 0 and determine do, 
then set n = 1 and determine a 1, etc. 

First we would like to write Eq. (31) in terms of binomial coefficients: 

n! = H(fc) fc!afc - (32) 

k 

The problem of solving implicit equations like this for <ik is called the inversion 
problem, and the technique we shall use applies to similar problems as well. 

The idea is based on the special case s = 0 of Eq. (23): 

^(1) (n)( _1 ) r_fc = (n - r) = <5 " r ’ inte S er n ’ inte g er r > (33) 

k 

The importance of this formula is that when n ^ r, the sum is zero; this enables 
us to solve our problem since a lot of terms cancel out as they did in Example 3: 

E<)i-r-=EE(K)H- 

-e«-eg) (:)<-»-- 

fc n 

— ^ ( k\ CLk^km — HI-* 
k 


' — kt 


(■ n — k)t 
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Notice how we were able to get an equation in which only one value a m appears, 
by adding together suitable multiples of Eq. (32) for n = 0 , 1 , We have now 




n>0 



E 

0 <n<m 


^ j 'jm—n 

(to — n)\ 


(~1) 

n! 

0 <n<m 


This completes the solution to Example 5 . Let us now take a closer look at 
the implications of Eq. (33): When r and to are nonnegative integers we have 

?(I) ( " irfe ( c °Co) +ci Ci) + ''' +cm (l) 


since the other terms vanish after summation. By properly choosing the coeffi- 
cients Cj, we can represent any polynomial in k as a sum of binomial coefficients 
with upper index k. We find therefore that 

£Q ) (— 1)’ fc (&o + hk H f b r k r ) = r\b r , integer r > 0, (34) 

k 

where bo + ■ ■ ■ + b r k r represents any polynomial whatever of degree r or less. 
(This formula will be of no great surprise to students of numerical analysis, 
since (I) (-l) r_fe /(:r + &) is the “rth difference” of the function f(x).) 

Using Eq. (34), we can immediately obtain many other relations that appear 
complicated at first and that are often given very lengthy proofs, such as 

^(1)0 r kt )^ k = tr ’ inte s err ^°- (35) 

k 

It is customary in textbooks such as this to give a lot of impressive examples 
of neat tricks, etc., but never to mention simple-looking problems where the 
techniques fail. The examples above may have given the impression that all 
things are possible with binomial coefficients; it should be mentioned, however, 
that in spite of Eqs. (10), (11), and (18), there seems to be no simple formula 
for the analogous sum 

UCMX7) +••■+(:). <*) 

k = 0 


when n < to. (For n = m the answer is simple; what is it? See exercise 36 .) 

On the other hand this sum does have a closed form as a function of n when 
to is an explicit negative integer; for example, 


E(?) = (-ir r ” +1 

k = 0 

There is also a simple formula 

£(:)(*- *)- 


k = 0 


TO /TO — 1 ' 
~2 


( „ ) 


( 37 ) 


( 38 ) 


for a sum that looks as though it should be harder, not easier. 
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How can we decide when to stop working on a sum that resists simplification? 
Fortunately, there is now a good way to answer that question in many important 
cases: An algorithm due to R. W. Gosper and D. Zeilberger will discover closed 
forms in binomial coefficients when they exist, and will prove the impossibility 
when they do not exist. The Gosper-Zeilberger algorithm is beyond the scope 
of this book, but it is explained in CMath §5.8. See also the book A = B by 
Petkovsek, Wilf, and Zeilberger (Wellesley, Mass.: A. K. Peters, 1996). 

The principal tool for dealing with sums of binomial coefficients in a sys- 
tematic, mechanical way is to exploit the properties of hypergeometric functions, 
which are infinite series defined as follows in terms of rising factorial powers: 


F 



a m 

bn 



__ n k k ~k 
a l * ‘ ‘ a m Z 


k> 0 


k\ 


(39) 


An introduction to these important functions can be found in Sections 5.5 and 
5.6 of CMath. See also J. Dutka, Archive for History of Exact Sciences 31 (1984), 
15-34, for historical references. 

The concept of binomial coefficients has several significant generalizations, 
which we should discuss briefly. First, we can consider arbitrary real values of the 
lower index k in ((,) ; see exercises 40 through 45. We also have the generalization 

(r\ _ {l-q r )(l-q r - l )...(l-q r - k+1 ) 

\k) q (l-g*)(l- g *-l)...(l-gl) ’ U j 

which becomes the ordinary binomial coefficient (”) = (”) when q approaches 
the limiting value 1; this can be seen by dividing each term in numerator and 
denominator by 1 — q. The basic properties of such “(/-normal coefficients” are 
discussed in exercise 58. 

However, for our purposes the most important generalization is the multi- 
nomial coefficient 


ki + k 2 + ■ ■ ■ + k n 
k\, k 2 , • • • ) k m 


(k\ + k 2 


k m )\ 


ki ! k 2 ! . . . k m ! 


integer ki > 0. 


(4i) 


The chief property of multinomial coefficients is the generalization of Eq. (13): 


(xi+x 2 -\ h x m ) n = 


E 


fci+&2H h km=n 


^1 5 ^2 ) • • • > 


ry* r rr* * nn 

Jo 1 Jo 9 ... Jo ^ 


*• (42) 


It is important to observe that any multinomial coefficient can be expressed in 
terms of binomial coefficients: 

/ fci +/S2 + - • -+fc m \ _ f T/l2 \ / k\ +k 2 +k 2 \ f ki+k 2 + - ■ ■+k m \ . 

V ki,k 2 , . . . ,k m J \ fci / \ k\-\-k 2 J \ ki + - ■ --\-k m -i ) 

so we may apply the techniques that we already know for manipulating binomial 
coefficients. Both sides of Eq. (20) are the trinomial coefficient 

( r y 

\k, m — k, r — m J 
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Table 2 

STIRLING NUMBERS OF BOTH KINDS 


0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

2 

0 

1 

1 

0 

0 

0 

0 

0 

0 

3 

0 

2 

3 

1 

0 

0 

0 

0 

0 

4 

0 

6 

11 

6 

1 

0 

0 

0 

0 

5 

0 

24 

50 

35 

10 

1 

0 

0 

0 

6 

0 

120 

274 

225 

85 

15 

1 

0 

0 

7 

0 

720 

1764 

1624 

735 

175 

21 

1 

0 

8 

0 

5040 

13068 

13132 

6769 

1960 

322 

28 

1 

n 

«} 

{?} 

ra 

a 

a 

f nl / n \/ n \/ n \ 
l 5 / l6Jl7il8j 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

2 

0 

1 

1 

0 

0 

0 

0 

0 

0 

3 

0 

1 

3 

1 

0 

0 

0 

0 

0 

4 

0 

1 

7 

6 

1 

0 

0 

0 

0 

5 

0 

1 

15 

25 

10 

1 

0 

0 

0 

6 

0 

1 

31 

90 

65 

15 

1 

0 

0 

7 

0 

1 

63 

301 

350 

140 

21 

1 

0 

8 

0 

1 

127 

966 

1701 

1050 

266 

28 

1 


For approximations valid when n is large, see L. Moser and M. Wyman, J. London Math. 
Soc. 33 (1958), 133-146; Duke Math. J. 25 (1958), 29-43; D. E. Barton, F. N. David, and 
M. Merrington, Biometrika 47 (1960), 439-445; 50 (1963), 169-176; N. M. Temme, Studies in 
Applied Math. 89 (1993), 233-243; H. S. Wilf, J. Combinatorial Theory A64 (1993), 344-349; 
H.-K. Hwang, J. Combinatorial Theory A71 (1995), 343-351. 

We conclude this section with a brief analysis of the transformation from 
a polynomial expressed in powers of r to a polynomial expressed in binomial 
coefficients. The coefficients involved in this transformation are called Stirling 
numbers, and these numbers arise in the study of numerous algorithms. 

Stirling numbers come in two flavors: We denote Stirling numbers of the first 
kind by [£], and those of the second kind by {£}. These notations, due to Jovan 
Karamata [ Mathematica (Cluj) 9 (1935), 164-178], have compelling advantages 
over the many other symbolisms that have been tried [see D. E. Knuth, AMM 
99 (1992), 403-422]. We can remember the curly braces in {”} because curly 
braces denote sets, and {£} is the number of ways to partition a set of n elements 
into k disjoint subsets (exercise 64). The other Stirling numbers [£] also have 
a combinatorial interpretation, which we will study in Section 1.3.3: [£] is the 
number of permutations on n letters having k cycles. 

Table 2 displays Stirling’s triangles, which are in some ways analogous to 
Pascal’s triangle. 
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Stirling numbers of the first kind are used to convert from factorial powers 
to ordinary powers: 


x- = x{x — 1 ) . . . (x — n + 1 ) 

r n 


LnJ 


x n — 


n 

n — 1 J 


n — 1 


+ ••• + (-1 r 




n 

Ik 


x k . 


(44) 


For example, from Table 2, 

H = ~r = ~^—(x 5 — 10x 4 + 35a : 3 — 50a; 2 + 24a:). 

V57 5! 120 v ' 

Stirling numbers of the second kind are used to convert from ordinary powers 
to factorial powers: 




x~ . 


(45) 


This formula was, in fact, Stirling’s original reason for studying the numbers 
{£} in his Methodus Differentialis (London: 1730). From Table 2 we have, for 
example, 

x 5 — x- + 10a:- + 25a:- + 15a;- + a:- 


-mm:) +i “G) + “(;) + G)- 

We shall now list the most important identities involving Stirling numbers. 
In these equations, the variables m and n always denote nonnegative integers. 

Addition formulas: 


\ n + 1 ] 

r n ' 

= n + 

n 

L m _ 

L m. 

_m — 1 . 

rn+l'l 

t m J 

f n 1 f n 

> =m\ [ + \ 

Irn ) Im — 


}• 


Inversion formulas (compare with Eq. ( 33 )) : 




r k 

m 


(-l)"- fc = s. 


Special values: 




(46) 


(47) 


(48) 

(49) 
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[thth [th. rrn. rn-^ 

(50) 


Expansion formulas: 

E 'nl / k ' 

.k. \m, 
k 


vr n i ( k \ = r n+1 i x"\ n+1 ]( ~ \ n i < \ 

VLfcJW Lm + 1 J ’ 44fc + lJW l } ~ Im ’ (5lj 

k k 

?(’ — {:}; (531 

E /m — n\ fm + n\ ( m + k' 1 r n 1 

\m + A:/ \ n + k ) l k J [n — ml’ 

E /m-n\/77i + n\rm + A;j r n "i ^4) 

K \m + k ) \ n + k J [ k \ In-mJ’ 

d ( 55 ) 


E 'fcln! rn + ll f fe l , . fn+li 

Ufc! = L + lJ’ ^{m} (m+1) ' = { m + 1 } (56) 

K^n k<n 

Some other fundamental Stirling number identities appear in exercises 1.2.6-61 
and 1.2. 7-6, and in Eqs. (23), (26), (27), and (28) of Section 1.2.9. 

Eq. (49) is just one instance of a general phenomenon: Both kinds of Stirling 
numbers [„”J and { n ” m } are polynomials in n of degree 2m, whenever m is a 
nonnegative integer. For example, the formulas for m = 2 and m = 3 are 


n ] / n\ ( n+l\ 

[n-2j-( 4 ) +2 ( 4 )’ 




u-g x:>cr). L-Mryo ©■ 

(57) 

Therefore it makes sense to define the numbers [ r 1 and j r j for arbitrary 

I / _ v Lr TThj L T 7Tt J *7 

real (or complex) values of r. With this generalization, the two kinds of Stirling 
numbers are united by an interesting duality law 

f n] r— to' 

{ml - [ —n J ’ (58 ) 
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which was implicit in Stirling’s original discussion. Moreover, Eq. (45) remains 
true in general, in the sense that the infinite series 

r 


= E{ r _*p 


r — k 


(59) 


converges whenever the real part of z is positive. The companion formula, 
Eq. (44), generalizes in a similar way to an asymptotic (but not convergent) 
series: 

m _ 

r 

' L r — k. 


= £ 


fc= 0 


(-1 ) k z r ~ k + 0(z r - m - 1 ) 


(60) 


(See exercise 65.) Sections 6.1, 6.2, and 6.5 of CMath contain additional informa- 
tion about Stirling numbers and how to manipulate them in formulas. See also 
exercise 4.7-21 for a general family of triangles that includes Stirling numbers 
as a very special case. 

EXERCISES 

1. [00] How many combinations of n things taken n — 1 at a time are possible? 

2. [00] What is Q? 

3. [00] How many bridge hands (13 cards out of a 52-card deck) are possible? 

4. [10] Give the answer to exercise 3 as a product of prime numbers. 

► 5. [05] Use Pascal’s triangle to explain the fact that ll 4 = 14641. 

► 6. [10] Pascal’s triangle (Table 1) can be extended in all directions by use of the 
addition formula, Eq. (9). Find the three rows that go on top of Table 1 (i.e., for 
r — —1, —2, and —3). 

7 . [12] If n is a fixed positive integer, what value of k makes (”) a maximum? 

8. [00] What property of Pascal’s triangle is reflected in the “symmetry condition,” 
Eq. (6)? 

9. [01] What is the value of (")? (Consider all integers n.) 

► 10. [M25] If p is prime, show that: 


a) = — (modulo p). 

b) eO (modulo p), for 1 < k < p — 1. 

c) k ^ = ( — l) fc (modulo p), for 0 < k < p — 1. 

d) ^ = 0 (modulo p), for 2 < k < p — 1. 

e) (E. Lucas, 1877.) 

(D-(Kj)ra 

f) If the p-ary number system representations of n and k are 

n = a T p r + • • • + fin p + do, 
k = b r p r -I h 61P + 60, 


then 
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► 11. [M20] (E. Rummer, 1852.) Let p be prime. Show that if p n divides 

(T) 

but p n+1 does not, then n is equal to the number of carries that occur when a is added 
to b in the p-ary number system. [Hint: See exercise 1.2.5-12.] 

12. [ M22 ] Are there any positive integers n for which all the nonzero entries in the 
nth row of Pascal’s triangle are odd ? If so, find all such n. 

13. [M13] Prove the summation formula, Eq. ( 10 ). 

14. [M21] Evaluate fc 4 . 

15. [ M15 ] Prove the binomial formula, Eq. ( 13 ). 

16. [Ml 5] Given that n and k are positive integers, prove the symmetrical identity 

► 17. [ M18 ] Prove the Chu-Vandermonde formula ( 21 ) from Eq. ( 15 ), using the idea 
that (1 + x) r+s = (1 + x) r (l + x) s . 

18. [M15] Prove Eq. ( 22 ) using Eqs. ( 21 ) and ( 6 ). 

19. [M18] Prove Eq. ( 23 ) by induction. 

20. [M20] Prove Eq. ( 24 ) by using Eqs. ( 21 ) and ( 19 ), then show that another use of 
Eq. ( 19 ) yields Eq. ( 25 ). 

► 21. [M05] Both sides of Eq. ( 25 ) are polynomials in s; why isn’t that equation an 
identity in s? 

22 . [ M20 ] Prove Eq. ( 26 ) for the special case s = n — 1 — r + nt. 

23. [M13] Assuming that Eq. ( 26 ) holds for ( r,s,t,n ) and (r, s - t, t, n - 1), prove 
it for (r, s + 1, t, n). 

24. [Ml 5] Explain why the results of the previous two exercises combine to give a 
proof of Eq. ( 26 ). 

25. [HM30] Let the polynomial A n (x,t) be defined as in Eq. ( 30 ). Let 2 = x t+1 - x l . 
Prove that A k (r,t)z k = x r , provided 2 is small enough. [Note: If t = 0, this result 
is essentially the binomial theorem, and this equation is an important generalization 
of the binomial theorem. The binomial theorem ( 15 ) may be assumed in the proof.] 
Hint: Start with the identity 

26. [ HM25 ] Using the assumptions of the previous exercise, prove that 



27. [HM21] Solve Example 4 in the text by using the result of exercise 25; and prove 
Eq. ( 26 ) from the preceding two exercises. [Hint: See exercise 17.] 
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28. [M25] Prove that 


n r T) {‘-.D-'zr.-'SY- 

k k> 0 


if n is a nonnegative integer. 

29. [M20] Show that Eq. (34) is just a special case of the general identity proved in 
exercise 1.2.3-33. 

► 30. [M24\ Show that there is a better way to solve Example 3 than the way used in 
the text, by manipulating the sum so that Eq. (26) applies. 

► 31. [M20] Evaluate 



m — r + s 
k 


W n + r - s\ 

( r + k \ 

A n — k ) 

\m + n) 


in terms of r, s, m, and n, given that m and n are integers. Begin by replacing 

( r + fc ) W r )( k \ 


32. [M20] Show that J^kH^ = A where x n is the rising factorial power defined in 
Eq. 1.2.5 (iq). 

33. [M20] (A. Vandermonde, 1772.) Show that the binomial formula is valid also 

when it involves factorial powers instead of the ordinary powers. In other words, prove 
that 

(*+»)“ = (*+»)*= E(fc)^ n - fe - 

k k 

34. [M23] ( Torelli’s sum.) In the light of the previous exercise, show that Abel’s 
generalization, Eq. (16), of the binomial formula is true also for rising powers: 

(x + y) n = -kz + l) k ~ 1 (y + kz) n ~ k . 


35. [M23] Prove the addition formulas (46) for Stirling numbers directly from the 
definitions, Eqs. (44) and (45). 

36. [ M10 ] What is the sum J2k (T) of the numb ers in each row of Pascal’s triangle? 
What is the sum of these numbers with alternating signs, Ylk 

37. [M10] From the answers to the preceding exercise, deduce the value of the sum 
of every other entry in a row, (") + (2) + (4) + ' ' ' • 

38. [ HM30 } (C. Ramus, 1834.) Generalizing the result of the preceding exercise, show 
that we have the following formula, given that 0 < k < m: 






1 J 7r \ j(n — 2k)n 

— > ( 2 cos — ) cos 

m \ m ) m 

0< j<m 


For example, 

a) + o + (") + -=K 2 ” +2 “^)- 

[Hint: Find the right combinations of these coefficients multiplied by mth roots of 
unity.] This identity is particularly remarkable when m > n. 
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39 . [M10] What is the sum of the numbers in each row of Stirling’s first 

triangle? What is the sum of these numbers with alternating signs? (See exercise 36.) 

40 . [HM1 7] The beta function B(x, y ) is defined for positive real numbers x, y by the 
formula B (x,y) = f* t x ~ 1 ( 1 - t) y ~ 1 dt. 

a) Show that B(x,l) = B(l,x) = l/x. 

b) Show that B(x + 1, y) + B(x, y + 1) = B(x, y). 

c) Show that B(x, y) = ((x + y)/y) B(x, y + 1 ). 

41 . [HM22] We proved a relation between the gamma function and the beta function 
in exercise 1.2.5—19, by showing that T m (x) = m x B(x,m + l), if m is a positive integer. 

a) Prove that 

B(X,2/) = f ^ ) +y) B{x ' V + m+1} - 


b) Show that 


B(x, y) 


r(»)r (y) 

r(x + y) ' 


42 . [ HM10 ) Express the binomial coefficient Q in terms of the beta function defined 
above. (This gives us a way to extend the definition to all real values of k.) 

43 . [ HM20 ] Show that B(l/2, 1/2) = 7 r. (Prom exercise 41 we may now conclude that 
T(l/ 2 ) = 0 r-) 


44 . [HM20] Using the generalized binomial coefficient suggested in exercise 42, show 
that 

45 . [HM21] Using the generalized binomial coefficient suggested in exercise 42, find 
linir-Kx, Q/r*. 

► 46 . [M21] Using Stirling’s approximation, Eq. 1 . 2 . 5 -( 7 ), find an approximate value 
( y V ')i assuming that both x and y are large. In particular, find the approximate 
size of ( 2 J“) when n is large. 

47 . [M21 ] Given that k is an integer, show that 


Give a simpler formula for the special case r = -1/2. 
► 48 . [ M25 ] Show that 


WMC- 1 )* _ n! 1 

^Vfc/fc + x x(x + l)...(x + n) x( n + 1 )’ 

if the denominators are not zero. [Note that this formula gives us the reciprocal of a 
binomial coefficient, as well as the partial fraction expansion of l/x(x + 1 ) . . . (x + n).\ 

49. [M20] Show that the identity (1 + x) r = (1 — x 2 ) r (l — x)~ r implies a relation on 
binomial coefficients. 

50 . [M20] Prove Abel’s formula, Eq. ( 16 ), in the special case x + y = 0 . 

51 . [ M21 ] Prove Abel’s formula, Eq. ( 16 ), by writing y = (x + y) — x, expanding the 
right-hand side in powers of (x + y), and applying the result of the previous exercise. 
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52. [HM11] Prove that Abel’s binomial formula (16) is not always valid when n is not 
a nonnegative integer, by evaluating the right-hand side when n = x = —1, y = z = 1. 

53. [M25] (a) Prove the following identity by induction on m, where m and n are 
integers: 


XXfcXn- J (nr “ (r + S)fc) = (m + 1 )(n " m) (m+l)(n-m)' 


(b) Making use of important relations from exercise 47, 


— 1/2 \ (~l)"/2n\ / 1/2 \ (—l)* 1 - 1 (2n\ (-l)"" 1 /2n-l\ 

n ) 2 2n \n/’ V n ) 2 2n (2n— 1) V n ) 2 2n-1 (2n— 1) V n ) 


show that the following formula can be obtained as a special case of the identity in 
part (a): 


u 


(2k — IN 

/ 2n — 2k\ 

V k ) 

V n — k ) 


-1 


2k 


n-m/2mU2n - 2m \ 1 / 2n \ 

2 n \ m ) \ n — m ) 2 \ n J 


(This result is considerably more general than Eq. (26) in the case r = —1, s = 0, 
t = -2.) 

54. [ M21 ] Consider Pascal’s triangle (as shown in Table 1) as a matrix. What is the 
inverse of that matrix? 

55. [ M21 ] Considering each of Stirling’s triangles (Table 2) as matrices, determine 
their inverses. 

56. [20] ( The combinatorial number system.) For each integer n = 0, 1, 2, . . . , 20, 
find three integers a, b, c for which n = (“) + (£) + (J) and a > b > c > 0. Can you see 
how this pattern can be continued for higher values of n? 

► 57. [M22] Show that the coefficient a m in Stirling’s attempt at generalizing the fac- 
torial function, Eq. 1.2.5-(i2), is 

^Fem>‘(I :>*• 

*:>i 


58. [M23] (H. A. Rothe, 1811.) In the notation of Eq. (40), prove the “q-nomial 
theorem” : 

(1 + a:)(l + qx) . . . (1 + q n_1 x) = q fc(fc_1)/2 a:' c . 

Also find q-nomial generalizations of the fundamental identities (17) and (21). 

59. [M25] A sequence of numbers A n k, n > 0, k > 0, satisfies the relations A n o = 1, 

Aok = S 0 k, A n k = + (”) for nk > 0. Find A n k- 

► 60. [M23] We have seen that (”) is the number of combinations of n things, k at 
a time, namely the number of ways to choose k different things out of a set of n. 
The combinations with repetitions are similar to ordinary combinations, except that we 
may choose each object any number of times. Thus, the list (1) would be extended to 
include also aaa, aab, aac, aad, aae, abb, etc., if we were considering combinations with 
repetition. How many fc-combinations of n objects are there, if repetition is allowed? 
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61. [M25] Evaluate the sum 



thereby obtaining a companion formula for Eq. (55). 

► 62. [. M23 ] The text gives formulas for sums involving a product of two binomial 
coefficients. Of the sums involving a product of three binomial coefficients, the following 
one and the identity of exercise 31 seem to be most useful: 



(The sum includes both positive and negative values of k.) Prove this identity. 

[Hint: There is a very short proof, which begins by applying the result of exercise 31.] 

63. [M30] If Z, m, and n are integers and n > 0, prove that 




;)©(*)(' 


‘H-CDL-;-,)- 


► 64. [M20] Show that { " } is the number of ways to partition a set of n elements 
into to nonempty disjoint subsets. For example, the set (1,2, 3, 4} can be partitioned 
into two subsets in {\} = 7 ways: {1,2,3}{4}; {1,2,4}{3}; {1,3, 4}{2}; {2,3,4}{1}; 
{1,2}{3,4}; {1, 3}{2, 4); {1,4}{2,3}. Hint: Use Eq. (46). 

65. [ HM35 ] (B. F. Logan.) Prove Eqs. (59) and (60). 

66. [HM30] Suppose x, y, and z are real numbers satisfying 

□ -CM.:,)- 

where x > n — 1, y > n — 1, z > n — 2, and n is an integer > 2. Prove that 

(n-l) - (n-l) + (n- 2 ) * and ° nly if » ^ 

(n+J-LiJ + O ^ and only if y < z. 

► 67. [M20] We often need to know that binomial coefficients aren’t too large. Prove 
the easy-to-remember upper bound 

(fc)-(f)’ when n > k > 0. 

68. [M25] (A. de Moivre.) Prove that, if n is a nonnegative integer, 

~ p } n ~ k \ k ~ np \ = )p r " Pl ( 1 -p) n+1_r " pl . 
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1.2.7. Harmonic Numbers 

The following sum will be of great importance in our later work: 

Hn=i+ \ + l + --- + l=itl' n ^°- (i) 

k = 1 


This sum does not occur very frequently in classical mathematics, and there is 
no standard notation for it; but in the analysis of algorithms it pops up nearly 
every time we turn around, and we will consistently call it H n . Besides H n , 
the notations h n and S n and ip(n + 1) + 7 are found in mathematical literature. 
The letter H stands for “harmonic,” and we speak of H n as a harmonic number 
because (1) is customarily called the harmonic series. Chinese bamboo strips 
written before 186 B.C. already explained how to compute H 10 = 7381/2520, as 
an exercise in arithmetic. [See C. Cullen, Historia Math. 34 (2007), 10-44.] 

It may seem at first that H n does not get too large when n has a large value, 
since we are always adding smaller and smaller numbers. But actually it is not 
hard to see that H n will get as large as we please if we take n to be big enough, 
because 

H 2 ^>l + ~. (2) 

This lower bound follows from the observation that, for m > 0, we have 


H 2 m T 

> H 2 m + 


1 


2 m + 1 2 m H 
1 1 
2m+l 2 m + 1 


H h 


2771+1 


2 m+l 


= Ho 


1 

2 * 


So as m increases by 1, the left-hand side of (2) increases by at least |. 

It is important to have more detailed information about the value of H n 
than is given in Eq. (2). The approximate size of H n is a well-known quantity 
(at least in mathematical circles) that may be expressed as follows: 


tf„ = l n » + 7 + + + 


120n 4 


252n b 


Here 7 = 0.5772156649 ... is Euler’s constant, introduced by Leonhard Euler in 
Commentarii Acad. Sci. Imp. Pet. 7 (1734), 150-161. Exact values of H n for 
small n, and a 40-place value for 7, are given in the tables in Appendix A. We 
shall derive Eq. (3) in Section 1.2.11.2. 

Thus H n is reasonably close to the natural logarithm of n. Exercise 7(a) 
demonstrates in a simple way that H n has a somewhat logarithmic behavior. 

In a sense, H n just barely goes to infinity as n gets large, because the similar 


sum 



(4) 


stays bounded for all n, when r is any real-valued exponent greater than unity. 
(See exercise 3.) We denote the sum in Eq. (4) by 
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When the exponent r in Eq. (4) is at least 2, the value of Hn 1 is fairly close 
to its maximum value H&, except for very small n. The quantity H& is very 
well known in mathematics as Riemann’s zeta function : 

1 


= CM = 

k> 1 


If r is an even integer, the value of ((r) is known to be equal to 

integer r/2 > 1, 


H £ = 2 \B, 


1 , (27r) r 


r! 


( 6 ) 


where B r is a Bernoulli number (see Section 1.2.11.2 and Appendix A). In 
particular, 




n 

IT 


h ( 4 ) = ZL 
00 90’ 


H (6) = H (S) = 

00 945 ’ 00 9450 ' 


(7) 


These results are due to Euler; for discussion and proof, see CMath, §6.5. 

Now we will consider a few important sums that involve harmonic numbers. 
First, 


^ H k — (n + 1 )H n - n. 


( 8 ) 


fc=i 


This follows from a simple interchange of summation: 

k ^ n n „ n , . 

n + l-j 

— 7 7 ~ — 7 

3 


n k 1 n n 1 n 

£ £j=£ £)=£ 

U — 1 A — 1 7 ■ — 1 I • J • 1 


fc=l 3 = 1 


j = 1 k=j 


1=1 


Formula (8) is a special case of the sum (m) H k • which we will now 

determine using an important technique called summation by parts (see exer- 
cise 10). Summation by parts is a useful way to evaluate J2a k b k whenever the 
quantities a k and (b k+ 1 — b k ) have simple forms. We observe in this case that 


and therefore 


hence 


to 

k = 1 


( k )=( k+i \-( k \ 

\mJ \m+ 1 / V m + 1 / ’ 

O-OK-iriJ-Lti)® 
Lhh-L'+ih 


k= 1 


k + 1 


k 

\17l 


1 

+ 1 \m) 
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Applying Eq. 1.2.6-(n) yields the desired formula: 

£( k )H k =( n l\)(H n+1 -^-). (9) 

\m+l/V m+1/ 

k — 1 

(This derivation and its final result are analogous to the evaluation of 

f n n m+1 ( l \ 1 

to + 1 V m + 1/ (m+1) 2 

using what calculus books call integration by parts.) 

We conclude this section by considering a different kind of sum, Yk (*•) x ' k , 

which we will temporarily denote by S n for brevity. We find that 


^n + 1 — 


E ((*) + (jfe- 1 )) a;fcfrfc_Sn + !B £ i (fc- 1 )* fc 1 { Hk ~ 1 + k :) 

= S n +xS n + ^-jJ2( k ) xk - 


k> 1 


Hence S n+ i = (x + 1)S„ + ((x + l) n+1 - l)/(n + 1), and we have 

S n+1 S n 1 1 

(x+l) n+1 (x + l) n n + 1 (n + l)(x + l) n+1 ' 

This equation, together with the fact that Si = x, shows us that 

/ = H n - ^ ,, 1 , )k - (l0) 

(x + l) n k(x + 1) R 

The new sum is part of the infinite series 1.2. 9-(i7) for ln(l/(l — 1 j(x + 1))) = 
ln(l + 1/x), and when x > 0, the series is convergent; the difference is 

k(x + l) fc < (n + l)(x + l) n+1 fy-' (x + l) fc ( n + l)(x + l)"x 

This proves the following theorem: 

Theorem A. If x > 0, then 

t{l) xkHk = (* + !)" - ln (i + ^)) + e ’ 

where 0 < e < l/(x(n + 1)) . | 


EXERCISES 

1. [01] What are H 0 , Hi, and H+1 

2. [ 13\ Show that the simple argument used in the text to prove that H^m > l + m/2 
can be slightly modified to prove that if2™ < 1 + m. 
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3. [M21] Generalize the argument used in the previous exercise to show that, for 
r > 1, the sum Illi 1 remains bounded for all n. Find an upper bound. 

► 4. [10] Decide which of the following statements are true for all positive integers n: 
(a) H n < Inn. (b) H n > Inn. (c) H n > Inn + 7 . 

5. [15] Give the value of H 10000 to 15 decimal places, using the tables in Appendix A. 

6 . [M15] Prove that the harmonic numbers are directly related to Stirling’s numbers, 
which were introduced in the previous section; in fact, 


Hn 


'n + 1 " 
. 2 . 



7. [M21] Let T(m,n ) = H m + H„ — H m n ■ (a) Show that when m or n increases, 
T(m, n) never increases (assuming that m and n are positive), (b) Compute the 
minimum and maximum values of T(m,n) for m,n > 0. 

8 . [HM18] Compare Eq. ( 8 ) with In k; estimate the difference as a function 

of n. 


► 9. [Ml 8] Theorem A applies only when x > 0 ; what is the value of the sum consid- 
ered when x = — 1 ? 

10. [M20] ( Summation by parts.) We have used special cases of the general method 
of summation by parts in exercise 1.2.4-42 and in the derivation of Eq. ( 9 ). Prove the 
general formula 


^ ^ ({Tfc+i a k )b k — a n b n a\b\ ^ b k ). 

l<fc<n l<fc<n 

► 11. [M21] Using summation by parts, evaluate 

► 12. [M10] Evaluate H^ 000 ^ correct to at least 100 decimal places. 

13. [M2 2] Prove the identity 


n if n 


<»-!)* 


(Note in particular the special case x — 0, which gives us an identity related to exercise 
1.2.6-48.) 

14. [Mae] Show that £Li H k /k = \ {Hi + H™), and evaluate H k /{k + 1). 

► 15. [M23] Express H k in terms of n and H n . 

16. [18] Express the sum 1 + 5 + • • • + 2n 1 _ 1 in terms of harmonic numbers. 

17. [M24] (E. Waring, 1782.) Let p be an odd prime. Show that the numerator of 
Hp-i is divisible by p. 

18. [M3 3] (J. Selfridge.) What is the highest power of 2 that divides the numerator 

of 1 1 4 - • * • — - — ? 

► 19. [M30] List all nonnegative integers n for which H n is an integer. [Hint: If H n 
has odd numerator and even denominator, it cannot be an integer.] 
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20. [ HM22 ] There is an analytic way to approach summation problems such as the one 
leading to Theorem A in this section: If f(x) = 5 ] fc>0 a kX k , and this series converges 
for x = xo, prove that 

f l{X °\- f(X ‘ y) dy. 

Jo l ~y 

21 . [M24] Evaluate ]T ]£ =1 Hk/(n + 1 — k). 

22. [M28] Evaluate 

► 23. [HM20] By considering the function T'(a:)/r(a:), show how we can get a natural 
generalization of H n to noninteger values of n. You may use the fact that T'(l) = — 7 , 
anticipating the next exercise. 

24. [HM21] Show that 

(Consider the partial products of this inflnite product.) 

1.2.8. Fibonacci Numbers 

The sequence 

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ..., ( 1 ) 

in which each number is the sum of the preceding two, plays an important role 
in at least a dozen seemingly unrelated algorithms that we will study later. The 
numbers in the sequence are denoted by F n , and we formally define them as 

F 0 = 0; Fi — 1; F„ +2 = F n+i + F n , n> 0. ( 2 ) 

This famous sequence was published in 1202 by Leonardo Pisano (Leonardo 
of Pisa), who is sometimes called Leonardo Fibonacci ( Filius Bonaccii , son of 
Bonaccio). His Liber Abaci (Book of the Abacus) contains the following exercise: 
“How many pairs of rabbits can be produced from a single pair in a year’s time?” 
To solve this problem, we are told to assume that each pair produces a new pair 
of offspring every month, and that each new pair becomes fertile at the age of 
one month. Furthermore, the rabbits never die. After one month there will be 
2 pairs of rabbits; after two months, there will be 3; the following month the 
original pair and the pair born during the first month will both usher in a new 
pair and there will be 5 in all; and so on. 

Fibonacci was by far the greatest European mathematician of the Middle 
Ages. He studied the work of al-Khwarizml (after whom “algorithm” is named, 
see Section 1.1) and he added numerous original contributions to arithmetic and 
geometry. The writings of Fibonacci were reprinted in 1857 [B. Boncompagni, 
Scritti di Leonardo Pisano (Rome, 1857-1862), 2 vols.; F n appears in Vol. 1, 283- 
285]. His rabbit problem was, of course, not posed as a practical application to 
biology and the population explosion; it was an exercise in addition. In fact, 
it still makes a rather good computer exercise about addition (see exercise 3); 
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Fibonacci wrote: “It is possible to do [the addition] in this order for an i nfin ite 
number of months.” 

Before Fibonacci wrote his work, the sequence (F n ) had already been dis- 
cussed by Indian scholars, who had long been interested in rhythmic patterns 
that are formed from one-beat and two-beat notes or syllables. The number of 
such rhythms having n beats altogether is F n+1 ; therefore both Gopala (before 
1135) and Hemacandra (c. 1150) mentioned the numbers 1 , 2, 3, 5, 8 , 13, 21 , 
34, ... explicitly. [See P. Singh, Historia Math. 12 (1985), 229-244; see also 
exercise 4.5.3-32.] 

The same sequence also appears in the work of Johannes Kepler, 1611, who 
was musing about the numbers he saw around him [J. Kepler, The Six-Cornered 
Snowflake (Oxford: Clarendon Press, 1966), 21]. Kepler was presumably unaware 
of Fibonacci’s brief mention of the sequence. Fibonacci numbers have often been 
observed in nature, probably for reasons similar to the original assumptions of 
the rabbit problem. [See Conway and Guy, The Book of Numbers (New York: 
Copernicus, 1996), 113-126, for an especially lucid explanation.] 

A first indication of the intimate connections between F n and algorithms 
came to light in 1837, when E. Leger used Fibonacci’s sequence to study the 
efficiency of Euclid’s algorithm. He observed that if the numbers m and n in 
Algorithm 1.1E are not greater than F k , step E2 will be executed at most k - 1 
times. This was the first practical application of Fibonacci’s sequence. (See 
Theorem 4.5.3F.) During the 1870s the mathematician E. Lucas obtained very 
profound results about the Fibonacci numbers, and in particular he used them to 
prove that the 39-digit number 2 127 -1 is prime. Lucas gave the name “Fibonacci 
numbers” to the sequence (F n ), and that name has been used ever since. 

We already have examined the Fibonacci sequence briefly in Section 1.2.1 
(Eq. ( 3 ) and exercise 4), where we found that <j> n ~ 2 <F n < if n is a positive 
integer and if 

*=|(l + v/5). ( 3 ) 

We will see shortly that this quantity, tj>, is intimately connected with the 
Fibonacci numbers. 

The number <j> itself has a very interesting history. Euclid called it the 
“extreme and mean ratio” ; the ratio of A to I? is the ratio of A + B to A, if the 
ratio of A to B is <f>. Renaissance writers called it the “divine proportion” ; and 
in the last century it has commonly been called the “golden ratio.” Many artists 
and writers have said that the ratio of 0 to 1 is the most aesthetically pleasing 
proportion, and their opinion is confirmed from the standpoint of computer 
programming aesthetics as well. For the story of <j>, see the excellent article “The 
Golden Section, Phyllotaxis, and Wythoff’s Game,” by H. S. M. Coxeter, Scripta 
Math. 19 (1953), 135-143; see also Chapter 8 of The 2nd Scientific American 
Book of Mathematical Puzzles and Diversions, by Martin Gardner (New York: 
Simon and Schuster, 1961). Several popular myths about <j> have been debunked 
by George Markowsky in College Math. J. 23 (1992), 2-19. The fact that the 
ratio F n+ i/F n approaches (j> was known to the early European reckoning master 
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Simon Jacob, who died in 1564 [see P. Schreiber, Histori a Math. 22 (1995), 
422-424], 

The notations we are using in this section are a little undignified. In much 
of the sophisticated mathematical literature, F n is called u n instead, and (f> is 
called r. Our notations are almost universally used in recreational mathematics 
(and some crank literature!) and they are rapidly coming into wider use. The 
designation <j> comes from the name of the Greek artist Phidias who is said 
to have used the golden ratio frequently in his sculpture. The notation F n is in 
accordance with that used in the Fibonacci Quarterly, where the reader may find 
numerous facts about the Fibonacci sequence. A good reference to the classical 
literature about F n is Chapter 17 of L. E. Dickson’s History of the Theory of 
Numbers 1 (Carnegie Inst, of Washington, 1919). 

The Fibonacci numbers satisfy many interesting identities, some of which 
appear in the exercises at the end of this section. One of the most commonly 
discovered relations, mentioned by Kepler in a letter he wrote in 1608 but first 
published by J. D. Cassini [Jfistoire Acad. Roy. Paris 1 (1680), 201], is 

Fn+lFn-l - Fl = (-1)”, ( 4 ) 

which is easily proved by induction. A more esoteric way to prove the same 
formula starts with a simple inductive proof of the matrix identity 


( F n + 1 

F n \ 

\-( l 

1 v 

V F n 

F n -J 

1 “ ll 

oJ 


We can then take the determinant of both sides of this equation. 

Relation (4) shows that F n and F n+ 1 are relatively prime, since any common 
divisor would have to be a divisor of (—1)". 

From the definition (2) we find immediately that 

F n + 3 = F n+ 2 + F n+ i = 2F n _|_i + F n ; F n+ 4 = 3F ra+ i + 2 F n \ 

and, in general, by induction that 

Fn+m = F m F n+ 1 + F m -iF n (6) 

for any positive integer to. 

If we take m to be a multiple of n in Eq. (6), we find inductively that 
F n k is a multiple of F n . 

Thus every third number is even, every fourth number is a multiple of 3, every 
fifth is a multiple of 5, and so on. 

In fact, much more than this is true. If we write gcd(m,n) to stand for the 
greatest common divisor of m and n, a rather surprising theorem emerges: 

Theorem A (E. Lucas, 1876). A number divides both F m and F n if and only 
if it is a divisor of Fd, where d = gcd(TO, n); in particular, 

gcd (F m ,F n ) — ^gcd(m,n)- 


(7) 
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Proof. This result is proved by using Euclid’s algorithm. We observe that 
because of Eq. ( 6 ) any common divisor of F m and F n is also a divisor of F n+m - 
and, conversely, any common divisor of F n+m and F n is a divisor of F m F n+1 . 
Since F n+1 is relatively prime to F n , a common divisor of F n+m and F n also 
divides F m . Thus we have proved that, for any number d, 

d divides F m and F n if and only if d divides F rn+n and F n . (8) 

We will now show that any sequence ( F n ) for which statement ( 8 ) holds, and 
for which F 0 = 0, satisfies Theorem A. 

First it is clear that statement ( 8 ) may be extended by induction on k to 
the rule 

d divides F m and F n if and only if d divides F m .- Xn and F n , 

where k is any nonnegative integer. This result may be stated more succinctly: 

d divides F rn mod n and F n if and only if d divides F rn and F n . ( 9 ) 

Now if r is the remainder after division of m by n, that is, if r = m mod n, 
then the common divisors of {F m ,F n } are the common divisors of {F n .F r }. It 
follows that throughout the manipulations of Algorithm 1.1E the set of common 
divisors of {F m , F n \ remains unchanged as m and n change; finally, when r - 0, 
the common divisors are simply the divisors of F 0 = 0 and F gcd(m n) . | 

Most of the important results involving Fibonacci numbers can be deduced 
from the representation of F n in terms of <j>, which we now proceed to derive. 
The method we shall use in the following derivation is extremely important, and 
the mathematically oriented reader should study it carefully; we will study the 
same method in detail in the next section. 

We start by setting up the infinite series 

G(z) = F 0 + F x z + F 2 z 2 + F 3 z 3 + F 4 z 4 H 

= z + z 2 + 2z 3 + 3z 4 + ■ ■ ■ . ( 10 ) 

We have no a priori reason to expect that this infinite sum exists or that the 
function G(z) is at all interesting — but let us be optimistic and see what we 
can conclude about the function G(z) if it does exist. The advantage of such a 
procedure is that G(z) is a single quantity that represents the entire Fibonacci 
sequence at once; and if we find out that G(z) is a “known” function, its 
coefficients can be determined. We call G(z) the generating function for the 
sequence (F n ). 

We can now proceed to investigate G(z) as follows: 

zG(z) = F 0 z + F x z 2 + F 2 z 3 + F 3 z 4 + ■■■, 
z 2 G(z ) = F 0 z 2 + F x z 3 + F 2 z 4 + • • • ; 

by subtraction, therefore, 

(1-z- z 2 )G(z ) = F 0 + (F x - F 0 )z + (F 2 - F 1 - F 0 )z 2 

+ (F 3 - Fa - F x )z 3 + (F 4 - F 3 - F 2 )z 4 + ■■■. 
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All terms but the second vanish because of the definition of F n , so this expression 
equals z. Therefore we see that, if G(z ) exists, 

G(z) = z/(l — z — z 2 ). (n) 

In fact, this function can be expanded in an infinite series in 2 (a Taylor series); 
working backwards we find that the coefficients of the power series expansion of 
Eq. (11) must be the Fibonacci numbers. 

We can now manipulate G(z) and find out more about the Fibonacci se- 
quence. The denominator 1 — 2 — z 2 is a quadratic equation with the two roots 
|(— 1 ± y/5 ); after a little calculation we find that G(z) can be expanded by the 
method of partial fractions into the form 

where 

< f > =1 - < f > = |(1 - \/ 5 ). 

The quantity 1/(1 — <f>z) is the sum of the infinite geometric series 1 + 4>z + 
4> 2 z 2 + ■ ■ ■ , so we have 

G(z) = — — (1 + <f>z + (f) 2 z 2 + 1 — cfiz — $i 2 z 2 — •■•). 

V 5 

We now look at the coefficient of z n , which must be equal to F n ; hence 

Fn = ~ (r - <T). ( 14 ) 

This is an important closed form expression for the Fibonacci numbers, first 
discovered early in the eighteenth century. (See D. Bernoulli, Comment. Acad. 
Sci. Petrop. 3 (1728), 85-100, §7; see also A. de Moivre, Philos. Trans. 32 (1722), 
162-178, who showed how to solve general linear recurrences in essentially the 
way we have derived (14).) 

We could have merely stated Eq. (14) and proved it by induction. However, 
the point of the rather long derivation above was to show how it would be 
possible to discover the equation in the first place, using the important method 
of generating functions, which is a valuable technique for solving a wide variety 
of problems. 

Many things can be proved from Eq. (14). First we observe that ^ is a 
negative number (—0.61803 . . . ) whose magnitude is less than unity, so (f> n gets 
very small as n gets large. In fact, the quantity <fi n /V 5 is always small enough 
so that we have 


(12) 

(* 3 ) 


F n = </>"/ Vh rounded to the nearest integer. 
Other results can be obtained directly from G(z); for example, 


G(z) 2 = - 


+ 


5 Vl 1 - 4>z) 2 (1 - $z) 


1 — 2 — 2 ' 


(l5) 


(l6) 
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and the coefficient of in G(z ) 2 is ELo F kF n - k . We deduce therefore that 

n 

F kFn-k = !((«+ 1 )(<A" + <T) - 2 F n + 1 ) 

— Q 

— 5 (( n T l){ P n + 2F n _i) — 2F n _j_i) 

= |(n-l)F„ + |nF„_ 1 . (i 7 ) 

(The second step in this derivation follows from the result of exercise 11.) 

EXERCISES 

1 . [ 10 ] What is the answer to Leonardo Fibonacci’s original problem: How many 
pairs of rabbits are present after a year? 

► 2. [ 20 ] In view of Eq. (15), what is the approximate value of F1000? (Use logarithms 
found in Appendix A.) 

3 . [ 25 ] Write a computer program that calculates and prints Fi through Fi 000 in 
decimal notation. (The previous exercise determines the size of numbers that must be 
handled.) 

► 4 . [ 1 J,] Find all n for which F n = n. 

5 . [ 20 ] Find all n for which F n = n 2 . 

6. [HM 10 ] Prove Eq. (5). 

► 7 . [ 15 ] If n is not a prime number, F n is not a prime number (with one exception). 
Prove this and find the exception. 

8. [ 15 ] In many cases it is convenient to define F n for negative n , by assuming that 
Fn + 2 = F n+ 1 + F n for all integers n. Explore this possibility: What is F_ x ? What is 
F- 2? Can F- n be expressed in a simple way in terms of F n ? 

9 . [M 20 ] Using the conventions of exercise 8, determine whether Eqs. (4), (6), (14), 
and (15) still hold when the subscripts are allowed to be any integers. 

10 . [ 15 ] Is 4> n /V 5 greater than F n or less than F n ? 

11 . [M 20 ] Show that <j> n = F n <t> + F n ^ and = F n fi + F n -i, for all integers n. 

► 12. [M 26 ] The “second order” Fibonacci sequence is defined by the rule 

Fo O5 F *1 = 1 , J - n ^_2 = F rt-f-l + Fn + F n . 

Express F n in terms of F n and F n +i. [Hint: Use generating functions.] 

► 13 . [M 22 ] Express the following sequences in terms of the Fibonacci numbers, when 
r, s, and c are given constants: 

a) ao = r , a± = s; a n +2 = a n+ 1 + a n , for n > 0. 

b) b 0 = 0, 61 = 1; b n+ 2 = 6 n+ i + b n + c, for n > 0. 

14 . [M 28 ] Let m be a fixed positive integer. Find a„, given that 

ao = 0, ai = 1; a n+ 2 = a„+ 1 + a n + (^), for n > 0. 

15 . [M 22 ] Let /(n) and g(n ) be arbitrary functions, and for n > 0 let 

a o 0, ai = 1, «n+2 — fln+i T a n + /(n); 

b 0 =0, bi = 1, b n+2 = b n+ 1 + b n + g(n); 

co = 0, ci = 1, c n + 2 = c n+1 + c n + xf(n) + yg(n). 

Express c„ in terms of x, y, a n , b n , and F n . 
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► 16. [M20] Fibonacci numbers appear implicitly in Pascal’s triangle if it is viewed from 

the right angle. Show that the following sum of binomial coefficients is a Fibonacci 
number: n , 

k = 0 

17 . [M2 4 } Using the conventions of exercise 8 , prove the following generalization of 
Eq. ( 4 ): F n + kFm-k — F n Fm = (— 1 ) n Fm-n-kFk- 

18. [20] Is F p + Pn+i always a Fibonacci number? 

► 19. [M27] What is cos36°? 

20. [M16] Express Y^k = 0 in terms of Fibonacci numbers. 

21. [ M25 ] What is YX=o F * xk? 

► 22. [ M20 ] Show that ^2 k (J^)F m + k is a Fibonacci number. 

23. [ M23 ] Generalizing the preceding exercise, show that ^-1 F m+k 

always a Fibonacci number. 

24. [HM20] Evaluate the n X n determinant 


/I 

-1 

0 

0 

... 0 

0 

°\ 

1 

1 

-1 

0 

... 0 

0 

0 

0 

1 

1 

-1 

. . . 0 

0 

0 

0 

0 

0 

0 

. . . 1 

1 

-1 

\0 

0 

0 

0 

... 0 

1 

1/ 


25. [M21] Show that . 

2"F„ = 2 E(fc) 5(,C " 1)/2 - 

k odd 

► 26. [M20] Using the previous exercise, show that F p = 5 (p~U / 2 (modulo p) if p is an 
odd prime. 

27. [ M20 ] Using the previous exercise, show that if p is a prime different from 5, then 
either F p - 1 or Fp+ 1 (not both) is a multiple of p. 

28. [M21] What is F„+i — 4>F n 7 

► 29. [M23] ( Fibonomial coefficients .) Edouard Lucas defined the quantities 

f n \ F n F n — 1 . . . F n —^-^-i "I r { F n —k+j \ 

\k)r~ F k F k - 1 ...F 1 ~ 1 = 1 V Fj ) 

in a manner analogous to binomial coefficients, (a) Make a table of (^) for 0 < k < 
n < 6 . (b) Show that is always an integer because we have 

► 30. [M3 8] (D. Jarden, T. Motzkin.) The sequence of mth powers of Fibonacci num- 
bers satisfies a recurrence relation in which each term depends on the preceding m+ 1 
terms. Show that 

E ( 7 X ( - 1)r - ,/21 ^ = 0 , if m > 0 . 

k 

For example, when m = 3 we get the identity F% — 2F p +i — 2F p+2 + F% + 3 = 0 . 
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31. [ M20 ] Show that F 2n ^mod 1 = 1- and F 2n+1 </>mod 1 = 0- 2 "- 1 . 

32. [M24] The remainder of one Fibonacci number divided by another is ± a Fibo- 
nacci number: Show that, modulo F n , 


Fm 


(-l) r+1 F n _ r , 
(-1 ) n F r , 
(~l) r+1+n F n . r , 


if m mod 4 = 0; 
if m mod 4=1; 
if m mod 4 = 2; 
if m mod 4 = 3. 


33. [JIM 2J f ] Given that z = tt/2 + i]n<f>, show that sinnz/sinz = i 1 ~ n F n . 

► 34. [M2 4] ( The Fibonacci number system.) Let the notation k » m mean that k > 
m + 2. Show that every positive integer n has a unique representation n = F kl + F k2 -|- 
b F kr , where ki > k 2 >•••» k r » 0. 

35. [M24] (A phi number system.) Consider real numbers written with the digits 0 
and 1 using base 0; thus (100.1)* = <t> 2 + Show that there are infinitely many 
ways to represent the number 1; for example, 1 = (.11)* = (.011111...)*. But if 
we require that no two adjacent Is occur and that the representation does not end 
with the infinite sequence 01010101. . . , then every nonnegative number has a unique 
representation. What are the representations of integers? 

► 36. [MS2] ( Fibonacci strings.) Let Sr = “a”, S 2 = “6”, and S n+2 = S n+1 S n , n > 0; 
m other words, S n+2 is formed by placing S n at the right of S n+1 . We have S 3 = “ba” , 
S 4 — bab , Sr, — babba , etc. Clearly S n has F n letters. Explore the properties of 
S n . (Where do double letters occur? Can you predict the value of the kth letter of S n ? 
What is the density of the 6’s? And so on.) 

► 37. [M35] (R. E. Gaskell, M. J. Whinihan.) Two players compete in the following 
game. There is a pile containing n chips; the first player removes any number of chips 
except that he cannot take the whole pile. From then on, the players alternate moves, 
each person removing one or more chips but not more than twice as many chips as the 
preceding player has taken. The player who removes the last chip wins. (For example, 
suppose that n = 11; player A removes 3 chips; player B may remove up to 6 chips, 
and he takes 1. There remain 7 chips; player A may take 1 or 2 chips, and he takes 2; 
player B may remove up to 4, and he picks up 1. There remain 4 chips; player A now 
takes 1; player B must take at least one chip and player A wins in the following turn.) 

What is the best move for the first player to make if there are initially 1000 chips? 

38. [35] Write a computer program that plays the game described in the previous 
exercise and that plays optimally. 

39. [M2 4] Find a closed form expression for a n , given that a 0 = 0, ai = 1, and 
a n + 2 = a n +i + 6 a n for n > 0. 

40. [M25] Solve the recurrence 


/(l) — 0; f(n) = o min^max(l + f(k), 2 + f(n — k)), for n > 1. 

► 41. [M25] (Yuri Matiyasevich, 1990.) Let f(x) = [x + 0” 1 ]. Prove that if n = 
Fk 1 + • • • + i'k r is the representation of n in the Fibonacci number system of exercise 34, 
then -Ffcj+i -| b F kr+1 = f(4>n). Find a similar formula for F kl _ 1 -| b F kr .- 1 . 

42. [ M26 ] (D. A. Klarner.) Show that if m and n are nonnegative integers, there is a 
unique sequence of indices fci » k 2 » • * • » k r such that 


m - F k x + F k 2 -\ b F kr , n = F kl+1 + F k2 + 1 + • • • + F kr+1 . 

(See exercise 34. The k's may be negative, and r may be zero.) 
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1.2.9. Generating Functions 

Whenever we want to obtain information about a sequence of numbers (a n ) = 
a o, ai, 02 , • ■ • , we can set up an infinite sum in terms of a “parameter” z, 

G(z) = do + d\z + d 2 2;2 + 1 ■ • = ^ ^ a n z n . (l) 

n> 0 

We can then try to obtain information about the function G. This function is a 
single quantity that represents the whole sequence; if the sequence ( a„ ) has been 
defined inductively (that is, if a n has been defined in terms of o 0 , or, . . . , a„_i) 
this is an important advantage. Furthermore, we can recover the individual 
values of o 0 , ai, . . . from the function G(z ), assuming that the infinite sum in 
Eq. (l) exists for some nonzero value of z, by using techniques of differential 
calculus. 

We call G(z ) the generating function for the sequence a 0 , a\, a 2 , .... The 
use of generating functions opens up a whole new range of techniques, and 
it broadly increases our capacity for problem solving. As mentioned in the 
previous section, A. de Moivre introduced generating functions in order to solve 
the general linear recurrence problem. De Moivre’s theory was extended to 
slightly more complicated recurrences by James Stirling, who showed how to 
apply differentiation and integration as well as arithmetic operations [ Methodus 
Differentialis (London: 1730), Proposition 15]. A few years later, L. Euler began 
to use generating functions in several new ways, for example in his papers on 
partitions [ Commentarii Acad. Sci. Pet. 13 (1741), 64-93; Novi Comment. Acad. 
Sci. Pet. 3 (1750), 125-169]. Pierre S. Laplace developed the techniques further 
in his classic work Theorie Analytique des Probabilities (Paris: 1812). 

The question of convergence of the infinite sum (l) is of some importance. 
Any textbook about the theory of infinite series will prove that: 

a) If the series converges for a particular value of z = zo, then it converges for 
all values of 2 with |z| < |zo|- 

b) The series converges for some z ^ 0 if and only if the sequence (\/|a n | ) is 
bounded. (If this condition is not satisfied, we may be able to get a conver- 
gent series for the sequence (a„/n!) or for some other related sequence.) 

On the other hand, it often does not pay to worry about convergence of 
the series when we work with generating functions, since we are only exploring 
possible approaches to the solution of some problem. When we discover the 
solution by any means, however sloppy, we may be able to justify the solu- 
tion independently. For example, in the previous section we used a generating 
function to deduce Eq. ( 14 ); yet once such an equation has been found, it is a 
simple matter to prove it by induction, and we need not even mention that we 
used generating functions to discover it. Furthermore one can show that most 
(if not all) of the operations we do with generating functions can be rigorously 
justified without regard to the convergence of the series. See, for example, E. T. 
Bell, Trans. Amer. Math. Soc. 25 (1923), 135-154; Ivan Niven, AMM 76 (1969), 
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871-889; Peter Henrici, Applied and Computational Complex Analysis 1 (Wiley, 
1974), Chapter 1. 

Let us now study the principal techniques used with generating functions. 

A. Addition. If G(z) is the generating function for (a„) = ao,^,... and 
H(z) is the generating function for (b n ) =b 0 ,b 1 ,..., then aG{z) + /3H(z) is the 
generating function for (aa n + /3b n ) = aa 0 + /3b 0 , aaq + pb x , . . . : 

a 'Yj a ' nZ + P "Y, b n z n — ^(aa n + /3b n )z n . ( 2 ) 

n>0 n> 0 n> 0 

B. Shifting. If G(z) is the generating function for (a n ) = a 0 ,ai,... then 
z m G(z) is the generating function for (a„_ m ) = 0, . , . , 0, a 0 , a u . . . : 

z m Y a n z n = Y a n-mZ n ■ ( 3 ) 

n> 0 n>m 

The last summation may be extended over all n > 0 if we regard a n = 0 for any 
negative value of n. 

Similarly, (G(z) - a 0 - a x z a TO _ 1 ^ m_1 )/z m is the generating function 

for (Un+m) ^m+li - ■ • • 

z 'y 1 a n z — y ] a n+m z n . (q) 

n>m n> 0 

We combined operations A and B to solve the Fibonacci problem in the 
previous section: G(z) was the generating function for (F n ), zG(z) for (F n -i), 
z G(z) for (F n _ 2 ), and (1 — z — z 2 )G(z) for (F n — F n _j — i 7 '„_ 2 ). Then, since 
Fn - F n _ 1 - F n _ 2 is zero when n > 2, we found that (1 - z - z 2 )G(z) is a 
polynomial. Similarly, given any linearly recurrent sequence, that is, a sequence 

where a n = cia n _i -\ |-c m a„_ m , the generating function will be a polynomial 

divided by (1 — c^z - • • • - c m z m ). 

Let us consider the simplest example of all: If G(z) is the generating func- 
tion for the constant sequence 1,1,1,..., then zG(z) generates 0,1,1,..., so 
(1 — z)G(z) — 1. This gives us the simple but very important formula 

■j— ; = l + z + z 2 + ---. ( 5 ) 

C. Multiplication. If G(z) is the generating function for a 0 ,Oi, . . . and H(z) 
is the generating function for b 0 , b 1 , . . . , then 

G(z)H(z) = (a 0 + aiz + a 2 z 2 -i )(b 0 + b x z + b 2 z 2 H ) 

= («oM + (ao^>i + aib 0 )z + (a 0 b 2 + aibi + a 2 b 0 )z 2 + • • • ; 

thus G{z)H{z) is the generating function for the sequence c 0 , c x , . . . , where 

n 

c n = y ] Qfc^n — k ■ 
k = 0 


( 6 ) 


1.2.9 


GENERATING FUNCTIONS 


89 


Equation ( 3 ) is a very special case of this. Another important special case occurs 
when each b n is equal to unity: 

y G(z) = ao + (ao + a\)z + (ao + (i\ + 02 )z 2 + • • • • (7) 

Here we have the generating function for the sums of the original sequence. 

The rule for a product of three functions follows from ( 6 ); F(z)G(z)H(z) 
generates d 0 , d\, d 2 , . . . , where 

d n = a ibj c k- ( 8 ) 

The general rule for products of any number of functions (whenever this is 
meaningful) is 


nE«.»‘=E'" E 

j>0 fc>0 


aofco a ifci 


(9) 


n> 0 fc 0 ,fci ,...>0 
fco + fci + ---=™ 


When the recurrence relation for some sequence involves binomial coeffi- 
cients, we often want to get a generating function for a sequence Co, ci, . . . defined 

by 


= E 


a kbn — k • 


( 10 ) 


In this case it is usually better to use generating functions for the sequences 
( a n /n \ ), (■ b n /n \ ), (c n /n!), since we have 


ao , «i . a 2 2 

0! + TT Z+ 2! 2 ' 


bo bi b 2 2 , 

0 ! + ll 2:+ 2! 2 + ' 


, Co Cl c 2 2 


where c n is given by Eq. ( 10 ). 


(11) 


D. Change of 2 . Clearly G(cz) is the generating function for the sequence 
ao, cai, c 2 a 2 , .... As a particular case, the generating function for 1, c, c 2 , c 3 , . . . 
is 1/(1 — cz ) . 

There is a familiar trick for extracting alternate terms of a series: 

|(G(z) + G(-z)) - a 0 + a 2 z 2 + a 4 z 4 + 

\{G{z) — G(-z)) — a\z + a 3 z 3 + a 5 z 5 + •••. 

Using complex roots of unity, we can extend this idea and extract every mth 
term: Let u = e 2,ri / m = cos(2n/m) + isin(2n/m); we have 


y, a n z n 

n mod m=r 


1 

m 


y oj~ kr G(Lo k z), 

0 <k<m 


0 < r < m. 


(13) 


(See exercise 14.) For example, if m = 3 and r — 1, we have u> — — \ + 
a complex cube root of unity; it follows that 

a\z + a±z 4 + a^z 1 + • ■ • = | (G(z) + w _ 1 G(wz) + c o~ 2 G{lo 2 z)) . 
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E. Differentiation and integration. The techniques of calculus give us further 

operations. If G(z ) is given by Eq. (1), the derivative is 


G'(z ) — cq + 2 d 2 Z + 3a 3 z~ + ■ ■ ■ = k + 1 )ak+i^ 


The generating function for the sequence ( na n ) is zG 1 (z). Hence we can combine 
the nth term of a sequence with polynomials in n by manipulating the generating 
function. 

Reversing the process, integration gives another useful operation: 

r z \ 1 1 

J G(t) dt = a 0 z + — ajz 2 + -a 2 z 3 + ■ • • = ^ -ak-\Z k . (15) 

k> 1 

As special cases, we have the derivative and integral of (5): 


(i -zy 


— 1 + 2z + 3z 2 + ■ ■ ■ = Yjk + 1 )z k . 


i 1 1 2 1 3 

In - z + - z 2 + -z J - 


—Ek- 


We can combine the second formula with Eq. (7) to get the generating function 
for the harmonic numbers: 

1 1 1 3 2 11 q , 

T^ 1 "— =*+r +¥* +--' = X>* ' (‘ 8 > 

fc> 0 

F. Known generating functions. Whenever it is possible to determine the 
power series expansion of a function, we have implicitly found the generating 
function for a particular sequence. These special functions can be quite useful 
in conjunction with the operations described above. The most important power 
series expansions are given in the following list. 

i) Binomial theorem. 


(i + , r = i + „ + hi_Li), 2 + 




When r is a negative integer, we get a special case already reflected in Eqs. (5) 
and (16): 




^ k '' ' » > 

There is also a generalization, which was proved in exercise 1.2.6-25: 


r , r i r — 2t — 1) 2 

x — 1 + rz H z 2 


r — kt\ r 


k 7 r — kt ’ 


* > (21) 


if x is the continuous function of z that solves the equation x t+1 = x l + z, where 
x = 1 when z = 0. 
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ii) Exponential series. 

exp 2 : = e z = 1 + 2 + ~z 2 + ■ ■ ■ = i z k . 

k> 0 

In general, we have the following formula involving Stirling numbers: 

iii) Logarithm series (see ( 17 ) and ( 18 )). 

ln(l + z) = z-^z 2 + ^z 3 = 


( 22 ) 


fc>i 


(1 - 2) m+1 


In 




■’sob 

k 

( 23 ) 

l) fc+1 A 
k 

( 24 ) 

'm + k\ 1 . 
k k ) Z ' 

( 25 ) 


Stirling numbers, as in ( 23 ), give us a more general equation: 


(‘“rb) 


= z n + 


1 


n+ 1 


n + 1 
n 


,n+ 1 




k I b 

Ln. fc! 


( 26 ) 


Further generalizations, including many sums of harmonic numbers, appear in 
papers by D. A. Zave, Inf. Proc. Letters 5 (1976), 75-77; J. Spiefi, Math. Comp. 
55 (1990), 839-863. 

iv) Miscellaneous. 


(1 - 2 ) (1 - 2 2 ) ... (1 - nz ) 


= 1 - TT-Z + — 2 


e 2 - 1 


12 


k 

( 27 ) 

k 

( 28 ) 

V - '' BkZ k 

^ k\ ' 

( 29 ) 


fc> 0 


The coefficients Bk that appear in the last formula are the Bernoulli numbers; 
they will be examined further in Section 1.2.11.2. A table of Bernoulli numbers 
appears in Appendix A. 

The next identity, analogous to ( 21 ), will be proved in exercise 2.3.4.4-29: 


x r — 1 + rz + 


r(r + 2 1 ) _ 2 , _ r ( r + kt ) k 1 _fc 


+ - = E 


fc >0 


k\ 


z , 


( 30 ) 


if x is the continuous function of 2 that solves the equation x = e zx , where x = 1 
when 2 = 0. Significant generalizations of ( 21 ) and ( 30 ) are discussed in exercise 
4.7-22. 
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G. Extracting a coefficient. It is often convenient to use the notation 

[z n ] G{z) (31) 

for the coefficient of z n in G(z). For example, if G(z) is the generating function 
in (1) we have [z n ] G{z) = a n and [z n }G(z)/{ 1 - z) = Ek=o a k- ° ne of the 
most fundamental results in the theory of complex variables is a formula of 
A. L. Cauchy [Exercices de Math. 1 (1826), 95-113 = (Euvres (2) 6, 124 145, 
Eq. (11)], by which we can extract any desired coefficient with the help of a 
contour integral: 

[z n ] G(z) = — i G( z ) dz , v 

1 J K 1 27rt/[ z|=r Z n + 1 ’ (32) 

if G{z) converges for z = z 0 and 0 < r < |z 0 |. The basic idea is that £. , z m dz 

. r J \z\=r 

is zero for all integers m except m = - 1, when the integral is 


f (re 10 ) 1 d(re 10 ) = i f dd — 2ni. 

J —IK J — 71 


Equation (32) is of importance primarily when we want to study the approximate 
value of a coefficient. 

We conclude this section by returning to a problem that was only partially 
solved in Section 1.2.3. We saw in Eq. 1.2.3-(i3) and exercise 1.2.3-29 that 


e x * x i=l(iL x ') 


l<i<j<r 


XiXj Xfc 


<k<n 




k = 1 / 1 


In general, suppose that we have n numbers xi, x 2 , . . . , x n and we want the 

sum 

h m= E *>!••• (33) 

l<Jl 

If possible, this sum should be expressed in terms of Si , S 2 , . . . , S m , where 

n 

s j = E x I> (34) 

k = 1 

the sum of jth powers. Using this more compact notation, the formulas above 
become h 2 = \S\ + |5 2 ; h 3 = |5f + §5^2 + | S 3 . 

We can attack this problem by setting up the generating function 

G(z) = 1 + h\z + h 2 z 2 + • • • = 'y ( hkZ k . (35) 
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By our rules for multiplying series, we find that 

G(z ) = (1 + x\Z + x 2 z 2 + • • • )(1 + x 2 z + x 2 z~ + •••)... (1 + x n z + x^z 2 + • • • ) 

(1 - 2 Ci 2 )(l ~ x 2 z) . . . (1 - x n z) ' ^ ^ 

So G(z) is the reciprocal of a polynomial. It often helps to take the logarithm 
of a product, and we find from (17) that 

1 . 1 


In G(z) = In 


1 — Xl z 

= ie¥ 


In- 


fc> 1 


1 - x n z 


x k „z k 


k> 1 


= E 


S k z k 


(37) 


k> 1 


Now In G(z) has been expressed in terms of the S’s; so all we must do to obtain 
the answer to our problem is to compute the power series expansion of G(z) 
again, with the help of (22) and (9): 


G( 2 ) = e lnG(z )=exp(]C 

k' 

S\z 


S k z k 


k> 1 
2 Jl 


=n* 

k> 1 


S k z k /k 


— ( 1 + S\Z + 


= E 


2 ! 


E 


S 2 z 2 S 2 z 4 


+ 


+ 


2 2 2 - 2 ! 

qki qk 2 qk 

°1 J 2 °m 


m > 0 ^ fei,fe 2 ,...,fe m >0 

fci + 2 /C 2 H 1 -mk rn =m 


l kl ki\ 2 k2 k 2 \ m km k m l 


(38) 


The parenthesized quantity is h m . This rather imposing sum is really not 
complicated when it is examined carefully. The number of terms for a particular 
value of m is p(m), the number of partitions of m (Section 1.2.1). For example, 
one partition of 12 is 

12 = 5 + 2 + 2 + 2+1; 


this corresponds to a solution of the equation ki + 2k 2 + • • • + 12fcj2 = 12, where 
kj is the number of j ’ s in the partition. In our example k\ = 1, k 2 — 3, k$ = 1, 
and the other fc’s are zero; so we get the term 


Si si s 5 

l 1 !! 2 3 3! 5 1 1 ! 


1 


1 Z 0 C QO q 

-1I-11 aim rlil n 1 2 ^ 


240 


as part of the expression for hi 2 . By differentiating (37) it is not difficult to 
derive the recurrence 


h n — — (Sih n -i + S 2 h n - 2 + • • • + S n ho), n > 1. (39) 

n 

An enjoyable introduction to the applications of generating functions has 
been given by G. Polya, “On picture writing,” AMM 63 (1956), 689-697; his 
approach is continued in CMath, Chapter 7. See also the book generatingfunc- 
tionology by H. S. Wilf, second edition (Academic Press, 1994). 
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A generating function is a clothesline 
on which we hang up a sequence of numbers for display. 

— H. S. WILF (1989) 


EXERCISES 

1. [ M12 ] What is the generating function for the sequence 2, 5, 13, 35,,.. = (2 n +3 n )? 

► 2. [Ml 3] Prove Eq. (n). 

3. [HM21 ] Differentiate the generating function (18) for (H n ), and compare this with 
the generating function for (J22 =0 H k ). What relation can you deduce? 

4. [M01] Explain why Eq. (19) is a special case of Eq. (21). 

5. [M20] Prove Eq. (23) by induction on n. 

► 6. [HM15] Find the generating function for 


E 

0<fc<n 


k(n — k ) 


differentiate it and express the coefficients in terms of harmonic numbers. 

7. [Ml 5] Verify all the steps leading to Eq. (38). 

8. [M23] Find the generating function for p(n), the number of partitions of n. 

9. [Mil] In the notation of Eqs. (34) and (35), what is h 4 in terms of S u S 2 , S 3 , 
and S 4 ? 


► 10. [M25] An elementary symmetric function is defined by the formula 


e m - 2^ x h ■ ■ ■ x im ■ 

l<jl <"-<3m <71 

(This is the same as h, n of Eq. (33), except that equal subscripts are not allowed.) Find 
the generating function for e m , and express e m in terms of the Sj in Eq. (34). Write 
out the formulas for ei, e 2 , e3, and e 4 . 

► 11. [ M25 ] Equation (39) can also be used to express the S’s in terms of the h’s: We 

Si — hi, S 2 = 2h 2 — hi, S3 = 3/13 — 3hih 2 + hf, etc. What is the coefficient of 
h^hrf.. . h in this representation of S m , when ki + 2k 2 d b ink m = m? 

► 12. [M20] Suppose we have a doubly subscripted sequence (a mn ) for m,n = 0, 1, ... ; 
show how this double sequence can be represented by a single generating function of 
two variables, and determine the generating function for ((”)}• 

13. [HM22] The Laplace transform of a function f(x) is the function 

P OO 

L /( s ) = / e“ s ' f(t)dt. 

Jo 

Given that fflo, 01, a 2 , . . . is an infinite sequence having a convergent generating function, 
let f(x) be the step function J2 k [0<fc<x]. Express the Laplace transform of f(x) 
in terms of the generating function G for this sequence. 

14. [HM21] Prove Eq. (13). 
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15 . [M28] By considering H(w) = ]T n>0 G„(z)w n , find a closed form for the gener- 
ating function 


G-w-iX” **>*=£: 


2k — n — 1 
k 



16 . [M22] Give a simple formula for the generating function G nr (z) = ’ g T lk a nk r z k , 
where a nkr is the number of ways to choose k out of n objects, subject to the condition 
that each object may be chosen at most r times. (If r = 1, we have (^) ways, and if 
r > k, we have the number of combinations with repetitions as in exercise 1.2.6-60.) 

17 . [ M25 ] What are the coefficients of 1/(1 — z) w if this function is expanded into a 
double power series in terms of both a and w? 

► 18. [M25] Given positive integers n and r, find a simple formula for the value of the 
following sums: (a) T,i< kl <k 2 <-<k r <n klk ^ ■ ■ ■ k G ( b ) J2i< kl <k 2 <-<k r <n ■ ■ ■ k f 

(For example, when n = 3 and r = 2 the sums are, respectively, 1-2 + 1- 3 + 2- 3 and 
11 + 1-2 + 1-3 + 2-2 + 2-3 + 3-3.) 


19 . [HM32] (C. F. Gauss, 1812.) The sums of the following infinite series are well 
known: 


Ui-U...= ln2; x 1 + 1 l + ... = 

Z 4 o O / 

1 1 1 77^3 1 . „ 

4 7 10 9 3 


7T 

4 ; 


Using the definition 



1 

n + x 


) 


found in the answer to exercise 1.2.7-24, these series may be written respectively as 


1_ 2 Hl/2 ’ 


4#!/4 + 4 ^ 3 / 4 ! 


l ~\ H i/e + Im- 


prove that, in general, H p / q has the value 


Q 7T V , „ 2 pk , . K 

cot -7r — ln2q + 2 > cos 7 r-lnsm- 7 r, 

V 2 a q q 

y y 0<k<q/2 


k 


when p and q are integers with 0 < p < q. [Hint: By Abel’s limit theorem the sum is 



1 

n + p/q 


) 


x p+ng 


Use Eq. (13) to express this power series in such a way that the limit can be evaluated.] 

20 . [M21] For what coefficients c mk is ^2 n>0 n m z n = CmfcZ V(l — z) k+x 2 

21 . [HM30] Set up the generating function for the sequence (n!) and study properties 
of this function. 

22 . [M21] Find a generating function G(z) for which 


lG(z) 


E 

fco + 2fci +4fc2+8fc3 + -- 


( r V r V r V r V. 

V k 0 ) V fci / V k 2 ) V kz ) 
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23. [M33\ (L. Carlitz.) (a) Prove that for all integers m > 1 there are polynomials 
fm (zi , . . . , z m ) and g m (zi Zm ) such that the formula 


fm (-2l , . . . , Zm ) g m ( Z\ , • . • , Z m ) 


fcl .,fc m >0 


is an identity for all integers n > r > 0 . 

(b) Generalizing exercise 15, find a closed form for the sum 


•**.)= E ( \ )( k \ ).■•( \ )»;■ 

, , . _ V n — k 2 ) \ n — k 3 ) \n - k\ ) 


^m 


in terms of the functions f m and g m in part (a). 

(c) Find a simple expression for S„(zi, . . . , z m ) when z\ — ■ ■ ■ = z m = z. 
24. [M2 2] Prove that, if G(z) is any generating function, we have 

j:(^)[z n - k ]G(z) k = [z n ](l + zG{z)r. 

k 


Evaluate both sides of this identity when G(z) is (a) 1 /( 1 -*); (b) (e 2 -l )/*. 

► 25. [M23] Evaluate the sum {V) Cn-T) ( — by simplifying the equivalent for- 
mula J2kl wk ] (1 “ 2w ) n [ z n ~ k ] (1 + z) 2n ~ 2k . 

26. [M40] Explore a generalization of the notation ( 31 ) according to which we might 
write, for example, [z 2 - 2 z 5 ] G(z) = a 2 - 2 a 5 when G(z) is given by ( 1 ). 


1.2.10. Analysis of an Algorithm 

Let us now apply some of the techniques of the preceding sections to the study 
of a typical algorithm. 

Algorithm M ( Find the maximum). Given n elements A[l], X[2 ], . . . , X[n], we 
will find to and j such that to = X[j] = maxi<j<„ X[i\, where j is the largest 
index that satisfies this relation. 

Ml. [Initialize.] Set j 4— n, k <— n — 1, m <— A[n], (During this algorithm we 
will have to = X[j] = max fc<i < n A[*].) 

M2. [All tested?] If k = 0, the algorithm terminates. 

M3. [Compare.] If X[k] < to, go to M5. 

M4. [Change to.] Set j <— k, to <— X[k}. (This value of to is a new current 
maximum.) 

M5. [Decrease k.] Decrease k by one and return to M2. | 

This rather obvious algorithm may seem so trivial that we shouldn’t bother 
to analyze it in detail; but it actually makes a good demonstration of the way 
in which more complicated algorithms may be studied. Analysis of algorithms 
is quite important in computer programming, because there are usually several 
algorithms available for a particular application and we would like to know which 
is best. 
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n — 1 


.No, 


Yes n 1 

— A ' A ' 

A 

v m>X[k ] 

n—l—A 



Fig. 9. Algorithm M. Labels on the arrows indicate the number of times each path 
is taken. Note that “Kirchhoff’s first law” must be satisfied: The amount of flow into 
each node must equal the amount of flow going out. 

Algorithm M requires a fixed amount of storage, so we will analyze only the 
time required to perform it. To do this, we will count the number of times each 
step is executed (see Fig. 9): 

Step number Number of times 


Ml 

1 

M2 

n 

M3 

n — 1 

M4 

A 

M5 

n — 1 


Knowing the number of times each step is executed gives us the information 
necessary to determine the running time on a particular computer. 

In the table above we know everything except the quantity A , which is the 
number of times we must change the value of the current maximum. To complete 
the analysis, we shall study this interesting quantity A. 

The analysis usually consists of finding the minimum value of A (for op- 
timistic people), the maximum value of A (for pessimistic people), the average 
value of A (for probabilistic people), and the standard deviation of A (a quanti- 
tative indication of how close to the average we may expect the value to be). 

The minimum value of A is zero; this happens if 

A'[n] = max X[k\. 

1 <fc<n 

The maximum, value is n — 1; this happens in case 

X[l] > X[2] > ■■■> X[n\. 

Thus the average value lies between 0 and n — 1. Is it |n? Is it y/n? 
To answer this question we need to define what we mean by the average; and 
to define the average properly, we must make some assumptions about the 
characteristics of the input data X[l], X[2], . . . , X[n]. We will assume that the 
X[k\ are distinct values, and that each of the n\ permutations of these values 
is equally likely. (This is a reasonable assumption to make in most situations, 
but the analysis can be carried out under other assumptions, as shown in the 
exercises at the end of this section.) 
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The performance of Algorithm M does not depend on the precise values 
of the X[k\; only the relative order is involved. For example, if n = 3 we are 
assuming that each of the following six possibilities is equally probable: 

Situation Value of A Situation Value of A 

X[l] < X[2] < X[3] 0 X[2] < X[3] < X[l] 1 

X[l] < X[3] < X[2\ 1 X[3] < X[l] < X[2] 1 

V[ 2 ] < X[l] < V[3] 0 X[3] < X[2] < X[l] 2 

The average value of A when n = 3 comes to (0 + 1 + 0 + 1 + 1 + 2)/6 = 5/6. 

It is clear that we may take X[l], X[2 ], . . . , X[n ] to be the numbers 1,2, ... ,n 
m some order; under our assumption we regard each of the nl permutations as 
equally likely. The probability that A has the value k will be 

Pnk = (number of permutations of n objects for which A = k)/n\. (i) 

For example, from our table above, p 30 = |, p 31 = |, p 32 = I. 

The average (“mean” or “expected”) value is defined, as usual, to be 

A n - Yl k Pn k- ( 2 ) 

k 

The variance V n is defined to be the average value of (A — A n ) 1 2 ; we have therefore 

Vn = “ A n ) 2 Pnk = k 2 p nk -2 A n ^2 kp n k + A 2 n y p nk 

k k k k 

= ^ k 2 p nk - 2 A n A n + A 2 = k 2 Pnk - A 2 n . ( 3 ) 

^ k 

Finally, the standard deviation a n is defined to be y/Vn- 

The significance of a n can perhaps best be understood by noting that, for 
all / >1, the probability that A fails to lie within ro n of its average value is 
less than 1/r 2 . For example, \A- A n \ > 2 u n with probability < 1/4. (Proof: 
Let p be the stated probability. Then if p > 0, the average value of (A — A n ) 2 
is more than p ■ ( ra n ) 2 + (1 - p) ■ 0; that is, V n > pr 2 V n .) This is usually called 
Chebyshev’s inequality, although it was actually discovered first by J. Bienayme 
[Comptes Rendus Acad. Sci. 37 (Paris, 1853), 320-321], 

We can determine the behavior of A by determining the probabilities p nk . 
It is not hard to do this inductively: By Eq. ( 1 ) we want to count the number of 
permutations on n elements that have A = k. Let this number be P nk = n\p nk . 

Consider the permutations Xl x 2 ... x n on { 1 , 2 , ... , n}, as in Section 1.2.5. 
If x x = n, the value of A is one higher than the value obtained om 2 ...i„; if 
X\ ^ n, the value of A is exactly the same as its value on x 2 . . . x n . Therefore 
we find that P nk = P( n -x)(k-x) + (n — l)P( n -\) k , or equivalently 

1 n — l 

Pnk — — P(n-l)(k-l) + 


n 


P(n— l)k- 


(4) 
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Fig. 10. Probability distribution for step M4, when n = 12. The mean is 58301/27720, 
or approximately 2.10. The variance is approximately 1.54. 


This equation will determine p n k if we provide the initial conditions 

Plk = ^ 0 /c l Pnk = 0 if k < 0. ( 5 ) 

We can now get information about the quantities p n k by using generating 
functions. Let 

G n (z) = PnO + PnlZ 4 ='^2PnkZ k - (6) 

k 

We know that A < n - 1 , so p n k = 0 for large values of k; thus G n (z) is actually 
a polynomial, even though an infinite sum has been specified for convenience. 
From Eq. ( 5 ) we have Gi(z) = 1; and from Eq. ( 4 ) we have 

G n (z) — — G n -i(z ) 4 G n -i(z) = G n -i(z). ( 7 ) 

n n n 

(The reader should study the relation between Eqs. ( 4 ) and ( 7 ) carefully.) We 
can now see that 

z + n — 1 z + n — 2 


G n (z)= Z + n - -Gn-l(z) 


G n - 2(2:) — 


1 


= — (z 4- n - 1 )(z 4 - n — 2) . . . (z + 1) 


1 / 2 4- n \ 

z + n V n J 


( 8 ) 


So G n (z ) is essentially a binomial coefficient! 

This function appears in the previous section, Eq. 1 . 2 . 9 -( 27 ), where we have 




_fc-i 


Therefore p n k can be expressed in terms of Stirling numbers: 

Pnk = [fc + l]/ nL 

Figure 10 shows the approximate sizes of p n k when n — 12. 


( 9 ) 
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Now all we must do is plug this value of p nk into Eqs. (2) and (3) and we 
have the desired average value. But this is easier said than done. It is, in fact, 
unusual to be able to determine the probabilities p nk explicitly; in most problems 
we will know the generating function G„(z), but we will not have any special 
knowledge about the actual probabilities. The important fact is that we can 
determine the mean and variance easily from the generating function itself. 

To see this, let’s suppose that we have a generating function whose coeffi- 
cients represent probabilities: 

G(z) = po + piz + p 2 z 2 H . 

Here pk is the probability that some event has a value k. We wish to calculate 
the quantities 

mean(G) = E kpk , var(G) = ^ fc 2 pfc - (mean(G)) 2 . (10) 

k k 

Using differentiation, it is not hard to discover how to do this. Note that 

G(l) = 1, (11) 

since G(l) = po +p 1 + p 2 -t is the sum of all possible probabilities. Similarly. 

since G'(z) = J2k ^ Pk z>i -1 , we have 

mean(G) = ^ kp k = G'(l). (12) 

k 

Finally, we apply differentiation again and we obtain (see exercise 2) 

var(G) = G"(l) + G'(l) — G'(l) 2 . (13) 

Equations (12) and (13) give the desired expressions of the mean and variance 
in terms of the generating function. 

In our case, we wish to calculate G’ n { 1) = A n . From Eq. (7) we have 

G' n {z) = —G n -i(z) — G' n _ 1 {z)\ 

G' n {l) = “ + G' n _ 1 (l). 

From the initial condition G' x (l) = 0, we find therefore 

= G^l) - H n - 1. (14) 

This is the desired average number of times step M4 is executed; it is approxi- 
mately Inn when n is large. [Note: The rth moment of A+ 1, namely the quantity 
+ 1 ) r Pnk, is [z n ] (1 — z) _1 {fe}(ln pT7) fc i an d it has the approximate 
value (lnn) r ; see P. B. M. Roes CACM 9 (1966), 342. The distribution of A 
was first studied by F. G. Foster and A. Stuart, J. Roy. Stat. Soc. B16 (1954), 
1 -22.) 

We can proceed similarly to calculate the variance V n . Before doing this, 
let us state an important simplification: 
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Theorem A. Let G and H be two generating functions with G( 1) — H( 1) = 1. 
If the quantities mean(G) and var(G) are defined by Eqs. (12) and (13), we have 

mean (GH) = mean(G) + mean (H)\ var (GH) = var(G) + var(H). (15) 

We will prove this theorem later. It tells us that the mean and variance of 
a product of generating functions may be reduced to a sum. | 

Letting Q n (z) = (z + n — l)/n, we have <5'„(1) = 1/n, Q”( 1) = 0; hence 

mean(Q n ) = -, var(Q„) = 

n n n z 

Finally, since G n (z) = IIfc=2 Qk( z )t it follows that 

n n 1 

mean(G n ) = ^ mean(Q fc ) = ^ - = H n - 1 
fc= 2 fc= 2 


var(G n ) = ^var(Q fc ) = ^ ^ J = H n - H^ ] . 

k = 2 fc=l ' ^ 

Summing up, we have found the desired statistics related to quantity A: 


A = ^min 0, ave H n — 1, max n — 1, dev y' H n — j . (16) 


The notation used in Eq. (16) will be used to describe the statistical character- 
istics of other probabilistic quantities throughout this book. 

We have completed the analysis of Algorithm M; the new feature that has 
appeared in this analysis is the introduction of probability theory. Elementary 
probability theory is sufficient for most of the applications in this book: The 
simple counting techniques and the definitions of mean, variance, and standard 
deviation already given will answer most of the questions we want to ask. More 
complicated algorithms will help us develop an ability to reason fluently about 
probabilities. 

Let us consider some simple probability problems, to get a little more 
practice using these methods. In all probability the first question that comes 
to mind is a coin-tossing problem: Suppose we flip a coin n times and there is a 
probability p that heads turns up after any particular toss; what is the average 
number of heads that will occur? What is the standard deviation? 

We will consider our coin to be biased; that is, we will not assume that 
p — L. This makes the problem more interesting, and, furthermore, every real 
coin is biased (or we could not tell one side from the other). 

Proceeding as before, we let p n k be the probability that k heads will occur, 
and let G n (z) be the corresponding generating function. We have clearly 


Pnk PP(n— l)(fc — 1) + QP(n — l)ki 


( 17 ) 


where q — 1 — p is the probability that tails turns up. As before, we argue from 
Eq. (17) that G n (z) = (q + pz)G n -i(z); and from the obvious initial condition 
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Gi(z) — q + pz we have 

G„(z) = (q + pz) n . (18) 

Hence, by Theorem A, 

mean(G n ) = n mean(Gi) = pn ; 
var(G n ) = n var(Gi) = (p - p 2 )n = pqn. 

For the number of heads, we have therefore 

(min 0, ave pn, max n, dev sjpqn ). (19) 

Figure 11 shows the values of p nk when p = f , n = 12. When the standard devi- 
ation is proportional to y/n and the difference between maximum and minimum 
is proportional to n, we may consider the situation “stable” about the average. 



0123456789 10 11 12 


Fig. 11. Probability distribution for coin-tossing: 12 independent tosses with a chance 
of success equal to 3/5 at each toss. 


Let us work one more simple problem. Suppose that in some process there 
is equal probability of obtaining the values 1 , 2 , . . . , n. The generating function 
for this situation is 


G(z)=-z + -z 2 + 
n n 


1 „ _ 1 z n+1 - x 
n n z — 1 


We find after some rather laborious calculation that 


n't \ _ nzU+1 ~(n+ 1 )z n + 1 

G (z) 7 7-75 ; 

n(z — 1 ) 2 

G „ (z) = n(n - l)^ 1 - 2 (n + l)(n - 1 )z n + n{n + 1 )z n ~ 1 - 2 

n(z — l) 3 


Now to calculate the mean and variance, we need to know G'(l) and G"(l); but 
the form in which we have expressed these equations reduces to 0/0 when we 
substitute z = 1 . This makes it necessary to find the limit as z approaches unity, 
and that is a nontrivial task. 

Fortunately there is a much simpler way to proceed. By Taylor’s theorem 
we have 


G(l + z) = G(l) + G'(l)z+^ 


,2 


(21) 
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therefore we merely have to replace z by z + 1 in ( 20 ) and read off the coefficients: 
G(l-M) = l (1 + * r ' 1 ~ 1 ~* = l + ^z+ ( ’ ,+ 1)( ’ , ~ 1 k ° + --.. 

n z 2 6 

It follows that G'(l) = |(n + 1), G"( 1) = |(n + l)(n — 1), and the statistics for 
the uniform distribution are 


min 1. 


n+ 1 (n + l)(n — 1) 

ave . max n, dev \ 

2 ’ ’ V 12 


( 22 ) 


In this case the deviation of approximately 0.289n gives us a recognizably un- 
stable situation. 

We conclude this section by proving Theorem A and relating our notions 
to classical probability theory. Suppose X is a random variable that takes on 
only nonnegative integer values, where X — k with probability p Then G(z) —■ 
Po + Pi z + P 2 Z 2 + • • • is called the probability generating function for X , and the 
quantity G{e lt ) = po+pie u +p 2 e 2lt +■ ■ ■ is conventionally called the characteristic 
function of this distribution. The distribution given by the product of two such 
generating functions is called the convolution of the two distributions, and it 
represents the sum of two independent random variables belonging to those 
respective distributions. 

The mean or average value of a random quantity X is often called its expected 
value, and denoted by EX. The variance of X is then EX 2 — (EX) 2 . Using this 
notation, the probability generating function for X is G(z) — E z x , the expected 
value of z x , in cases when X takes only nonnegative integer values. Similarly, 
if X is a statement that is either true or false, the probability that X is true is 
Pr(X) = E[X], using Iverson’s convention (Eq. 1.2.3-(i6)). 

The mean and variance are just two of the so-called semi-invariants or cu- 
mulants introduced by T. N. Thiele in 1889 [see A. Hald, International Statistical 
Review 68 (2000), 137-153]. The semi-invariants /tj , k 2 , K 3 , . . . are defined by 
the rule 

K 2 t 2 K 3 t 3 


Kit. 

IT 


+ 


2 ! 


+ 


3! 


+ ■■■ = lnG(e*). 


(23) 


We have 


in particular, 


= — lnG(e 
dt n v 


« 1 = 


‘GV) 


t = 0 


G(e‘) 


t = 0 


= G'(1) 


because G(l) = ~^2 k pk = 1, and 


K 2 


e^G'^e*) 

W) 


e t G'(e t ) 

W) 


e 2< G'(e 4 ) 2 

G(e t ) 2 


t - 0 


= G"(l) + G'(l) - G'(l) 2 . 


Since the semi-invariants are defined in terms of the logarithm of a generating 
function, Theorem A is obvious, and, in fact, it can be generalized to apply to 
all of the semi-invariants. 
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A normal distribution is one for which all semi-invariants are zero except 
the mean and variance. In a normal distribution, we can improve significantly 
on Chebyshev’s inequality: The probability that a normally distributed random 
value differs from the mean by less than the standard deviation is 



that is, about 68.268949213709% of the time. The difference is less than twice 
the standard deviation about 95.449973610364% of the time, and it is less than 
three times the standard deviation about 99.730020393674% of the time. The 
distributions specified by Eqs. (8) and (18) are approximately normal when n is 
large (see exercises 13 and 14). 

We often need to know that a random variable is unlikely to be much larger 
or smaller than its mean value. Two extremely simple yet powerful formulas, 
called the tail inequalities, provide convenient estimates of such probabilities. 
If X has the probability generating function G(z), then 

Pr(X < r) < x~ r G(x ) for 0 < x < 1; (24) 

Pr(X > r) < x~ r G(x) for x > 1. (25) 

The proofs are easy: If G(z) = p 0 +P 1 Z + p 2 z 2 -| , we have 

P f (^ <• r ) — Po +Pi H P P[r-J < x r p 0 +x 1 ~ r pi + • ■ • + xL r J -r p^ r j < x~ r G(x) 

when 0 < x < 1, and 

Pr(X > r) =p M +P|- r ]+i -I < x M_ >|- r ] + x |V1+1_r j>|y| +1 -| < x~ r G(x) 

when x > 1. By choosing values of x that minimize or approximately minimize 
the right-hand sides of (24) and (25), we often obtain upper bounds that are 
fairly close to the true tail probabilities on the left-hand sides. 

Exercises 21-23 illustrate the tail inequalities in several important cases. 
These inequalities are special cases of a more general principle first pointed out 
by A. N. Kolmogorov in his book Grundbegriffe der Wahrscheinlichkeitsrechnung 
(Springer, 1933): If /(f) > s > 0 for all t > r, then Pr(X > r) < s^E/pf) 
whenever E f(X) exists. We obtain (25) when f(t) = x* and s = x r . 

EXERCISES 

1. [10] Determine the value of p n 0 from Eqs. (4) and (5) and interpret this result 
from the standpoint of Algorithm M. 

2. [HM16] Derive Eq. (13) from Eq. (10). 

3. [Ml 5] What are the minimum, maximum, average, and standard deviation of 
the number of times step M4 is executed, if we are using Algorithm M to find the 
maximum of 1000 randomly ordered, distinct items? (Give your answer as decimal 
approximations to these quantities.) 

4. [M10] Give an explicit, closed formula for the values of p nk in the coin-tossing 
experiment, Eq. (17). 

5. [Ml 3] What are the mean and standard deviation of the distribution in Fig. 11? 


1 . 2.10 


ANALYSIS OF AN ALGORITHM 105 


6. [ HM27 ] We have computed the mean and the variance of the important probabil- 
ity distributions (8), (18), (20). What is the third semi-invariant, K3, in each of those 
cases? 

► 7. [M2 7] In our analysis of Algorithm M, we assumed that all the X[k] were distinct. 
Suppose, instead, that we make only the weaker assumption that A[l], A' [2], ... ,X[n] 
contain precisely m distinct values; the values are otherwise random, subject to this 
constraint. What is the probability distribution of A in this case? 

► 8. [M20] Suppose that each X[fc] is taken at random from a set of M distinct 
elements, so that each of the M n possible choices for X[l], X[2], . . . , X[n] is considered 
equally likely. What is the probability that all the X [k] will be distinct? 

9. [ M25 ] Generalize the result of the preceding exercise to find a formula for the 
probability that exactly m distinct values occur among the A’s. Express your answer 
in terms of Stirling numbers. 

10. [ M20 ] Combine the results of the preceding three exercises to obtain a formula 
for the probability that A = k under the assumption that each X is selected at random 
from a set of M objects. 


► 11. [M15] What happens to the semi-invariants of a distribution if we change G(z) 
to F(z) = z n G(z)? 

12. [HM21] When G(z) = po + piz + P 2 Z 2 + • • • represents a probability distribution, 
the quantities M n = J2k k n Pk and m n = ][[]*. (fc — M\) n pk are called the “nth moment” 
and “nth central moment,” respectively. Show that G{e t ) = 1 + Mit + M2t 2 /2\ + • • • ; 
then use Arbogast’s formula (exercise 1.2.5-21) to show that 


E 


(-D 


+ &2 H — 1 


n! (fci + k 2 + ■ 


fcl ,fc2 >0 

fcl+2fc2H =n 


fci! 1 ! fcl fc2 ! 2P 2 . . . fc„! n\ k 


M* 2 . . . M*" 


In particular, = Mi, «2 = M 2 — Ml (as we already knew), K3 = M3 — 3M1M2 + 2Ml , 
and k 4 = M 4 — 4MiM 3 + 12 Mf M 2 — 3M| — 6 M 4 . What are the analogous expressions 
for K n in terms of the central moments m 2 , m 3, . . . , when n > 2? 

13. [HMS8] A sequence of probability generating functions G n (z) with means p, n and 
deviations a n is said to approach a normal distribution if 


lim e - it ^ /cr "G n (e it/c ") = e- t2/2 


for all real values of t. Using G n {z) as given by Eq. (8), show that G n {z) approaches 
a normal distribution. 

Note: “Approaching the normal distribution,” as defined here, can be shown to 
be equivalent to the fact that 


lim Pr 

n — foo 


( 


X n Pn 

On 



1 

\/ 27 r 



dt, 


where X n is a random quantity whose probabilities are specified by G n (z). This is 
a special case of P. Levy’s important “continuity theorem,” a basic result in mathe- 
matical probability theory. A proof of Levy’s theorem would take us rather far afield, 
although it is not extremely difficult [for example, see Limit Distributions for Sums 
of independent Random Variables by B. V. Gnedenko and A. N. Kolmogorov, tr. by 
K. L. Chung (Reading, Mass.: Addison- Wesley, 1954)]. 
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14. [HM30] (A. de Moivre.) Using the conventions of the previous exercise, show that 
the binomial distribution G n (z) given by Eq. (18) approaches the normal distribution. 

15. [HM23] When the probability that some quantity has the value k is e~^(p k /k\), 
it is said to have the Poisson distribution with mean p. 

a) What is the generating function for this set of probabilities? 

b) What are the values of the semi-invariants? 

c) Show that as n — > oo the Poisson distribution with mean np approaches the normal 
distribution in the sense of exercise 13. 

16. [M25] Suppose X is a random variable whose values are a mixture of the proba- 
bility distributions generated by gi(z), g?{z), g r (z ), in the sense that it uses g k (z) 
with probability p k , where p\ + P 2 -¥■■•+ p r — 1- What is the generating function 
for XI Express the mean and variance of X in terms of the means and variances of gi, 
92, ■■■, Sr- 

► 17. [M2 7] Let f(z) and g(z) be generating functions that represent probability dis- 
tributions. 

a) Show that h(z) = g{f(z)) is also a generating function representing a probability 
distribution. 

b) Interpret the significance of h{z) in terms of f(z) and g(z). (What is the meaning 
of the probabilities represented by the coefficients of h(z)?) 

c) Give formulas for the mean and variance of h in terms of those for / and g. 

18. [M28] Suppose that the values taken on by X[l], X[2], . . . , X[n] in Algorithm M 
include exactly k\ ones, k 2 twos, . . . , k n n’s, arranged in random order. (Here 

1 1^2 1 t k n = n. 

The assumption in the text is that k x = k 2 = ■ ■ ■ = k n = 1.) Show that in this 
generalized situation, the generating function (8) becomes 

( k n — \z T k n \ / k n — 2 Z T k n — \ T k n \ f k\z -f- k 2 T • • • -I k n \ 

\ k n — i -f- k n J \ k n — 2 T k n ~ i k n J y k\ -f- k2 T * * * T k n J 

using the convention 0/0 = 1. 

19. [ M21 ] If a k > aj for 1 < j < k, we say that a k is a left-to-right maximum of the 

sequence ai <12 ■ ■ ■ a n . Suppose ai <3,2 . . . a n is a permutation of { 1 , 2, . . . , n}, and let 

bi b 2 • • • b n be the inverse permutation, so that a k = l if and only if b t = k. Show that 
a k is a left-to-right maximum of ai o 2 ... a n if and only if k is a right-to-left minimum 
of bi b 2 ... b n . 

► 20. [ M22 ] Suppose we want to calculate max{|ai - 61 1, |a 2 - b 2 |, . . . , \a n - 6 n |} when 
b\ < b 2 < ■ ■ ■ < b n . Show that it is sufficient to calculate maxfmi, mn}, where 

tul = maxjot — b k \ a k is a left-to-right maximum of aia 2 . . . a„} , 
niR = ma x{b k — a k \ a k is a right-to-left minimum of oia 2 . . . a n } ■ 

[Thus, if the a’s are in random order, the number of k’s for which a subtraction must 
be performed is only about 2 Inn.] 

► 21. [HM21] Let X be the number of heads that occur when a random coin is flipped 
n times, with generating function (18). Use (25) to prove that 

Pr(X > n(p + e)) < e~ e2n/{2q) 

when 6 > 0, and obtain a similar estimate for Pr(X < n(p — e)). 
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► 22 . [HM22] Suppose A' has the generating function (<ji + piz){q2 + P2z) ■ ■ ■ (</n +Pnz), 
where pk + qk = 1 for 1 < k < n. Let p = E X = pi + p 2 + • ■ • + p n - (a) Prove that 

Pr(X < pr) < (r _r e r_1 )* 1 , when 0 < r < 1; 

Pr(A' > pr ) < (r~ r e ~ 1 )^ , when r > 1. 

(b) Express the right-hand sides of these estimates in convenient form when r ft; 1 . 

(c) Show that if r is sufficiently large we have Pr(A' > pr) < 2 _Mr . 

23. [HM23] Estimate the tail probabilities for a random variable that has the negative 
binomial distribution generated by (q — pz)~ n , where q = p + 1. 

*1.2.11. Asymptotic Representations 

We often want to know a quantity approximately, instead of exactly, in order to 
compare it to another. For example, Stirling’s approximation to n! is a useful 
representation of this type, when n is large, and we have also made use of the 
fact that H n ss Inn + 7. The derivations of such asymptotic formulas generally 
involve higher mathematics, although in the following subsections we will use 
nothing more than elementary calculus to get the results we need. 

*1.2.11.1. The O- notation. Paul Bachmann introduced a very convenient 
notation for approximations in his book Analytische Zahlentheorie (1894). It is 
the O-notation, which allows us to replace the sign by “=” and to quantify 
the degree of accuracy; for example, 

H n = lnn + 7 + O^ij. (1) 

(Read, U H sub n equals the natural log of n plus Euler’s constant [pronounced 
‘Oiler’s constant’] plus big-oh of one over 

In general, the notation 0(/(n)) may be used whenever f(n) is a function 
of the positive integer n; it stands for a quantity that is not explicitly known, 
except that its magnitude isn’t too large. Every appearance of 0(/(n)) means 
precisely this: There are positive constants M and no such that the number x n 
represented by 0(/(n)j satisfies the condition ja: n | < M|/(n)|, for all integers 
n > no- We do not say what the constants M and nq are, and indeed those 
constants are usually different for each appearance of O. 

For example, Eq. (1) means that \H n — Inn — 7) < M/n when n > no- 
Although the constants M and no are not stated, we can be sure that the quantity 
0(l/n) will be arbitrarily small if n is large enough. 

Let’s look at some more examples. We know that 

l 2 + 2 2 + hn 2 = |n(n + |)(n + 1) = \n z + \n 2 + \n\ 

l 2 + 2 2 + ■ • • + n 2 — 0(n 4 ), (2) 

l 2 + 2 2 + • • • + n 2 = 0(n 3 ), (3) 

l 2 + 2 2 + bn 2 = |n 3 + 0(n 2 ). (4) 


so it follows that 
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Equation (2) is rather crude, but not incorrect; Eq. (3) is a stronger statement: 
and Eq. (4) is stronger yet. To justify these equations we shall prove that if 
P{n) = a 0 + a x n + ■ ■ ■ + a m n rn is any polynomial of degree m or less, then we 
have P{n) = 0(n m ). This follows because 

l-^( n )l - l a o| + K| n 4 f |a m | n m = (|a 0 | /n m + \a 1 \/n m ~ 1 H f \a m \)n m 

< (|a 0 | + |ai| + b |a m j)n m , 

when n> 1. So we may take M = |a 0 | + |ai| H 1- \a m \ and n 0 = 1. Or we 

could take, say, M = |a 0 |/2 m + \a l \/2 m - 1 + • • • + \a m \ and n 0 = 2. 

The O-notation is a big help in approximation work, since it describes briefly 
a concept that occurs often and it suppresses detailed information that is usually 
irrelevant. Furthermore, it can be manipulated algebraically in familiar ways, 
although certain important differences need to be kept in mind. The most impor- 
tant consideration is the idea of one-way equalities: We write |n 2 + n = 0(n 2 ), 
but we never write 0(n 2 ) = \n 2 + n. (Or else, since \n 2 = 0(n 2 ), we might 
come up with the absurd relation I71 2 = |n 2 + n.) We always use the convention 
that the right-hand side of an equation does not give more information than the 
left-hand side- the right-hand side is a “crudification” of the left. 

This convention about the use of “=’ may be stated more precisely as 
follows: Formulas that involve the 0(/(n))-notation may be regarded as sets 
of functions of n. The symbol () (/ (n) 'j stands for the set of all functions q of 
integers such that there exist constants M and n 0 with \g(n)\ < M\f(n)\ for 
all integers n > uq. If S and T are sets of functions, then S y If denotes the 
set {g + h | g E S and h £ T}; we define S + c, S - T, S ■ T, log S, etc., in a 
similar way. If a(n) and /3(n) are formulas that involve the O-notation, then 
the notation a(n) = 0 (n) means that the set of functions denoted by a(n) is 
contained in the set denoted by 0 {n). 

Consequently we may perform most of the operations we are accustomed to 
doing with the “=” sign: If a(n) = 0 (n) and 0 {n) = 7 (n), then a(n) = 7 (n). 
Also, if a(n) = 0 (n) and if S(n) is a formula resulting from the substitution of 
0(n) for some occurrence of a(n) in a formula 7 (n), then 7 (n) = S(n). These two 
statements imply, for example, that if g{x x ,x 2 , ■ . . , x m ) is any real function what- 
ever, and if a k (n) = 0 k (n) for 1 < k < m, then g( ai (n), a 2 (n), . . . , arn {n)) = 
g{0i(n),0 2 (n),...,0 m {n)). 

Here are some of the simple operations we can do with the O-notation: 


f(n) = 0(/(n)), (5) 

c • 0(/(n)) = 0(/(n)), if c is a constant, (6) 

0(/(n))+0(/(n)) =0(/(n)), ( 7 ) 

0(0(/(n)))=0(/(n)), (8) 

0 (f(n)) 0 (g(n)) = 0 (f(n)g(n)), ( 9 ) 

0 (f(n)g(n))=f(n) 0 (g(n)). (10) 
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The O-notation is also frequently used with functions of a complex variable 2, 
in the neighborhood of 2 = 0 . We write 0 (f(z )) to stand for any quantity g(z) 
such that |g(2)| < M \f[z)\ whenever \z\ < r. (As before, M and r are unspecified 
constants, although we could specify them if we wanted to.) The context of O- 
notation should always identify the variable that is involved and the range of 
that variable. When the variable is called n, we implicitly assume that 0 (/(n)) 
refers to functions of a large integer n; when the variable is called 2, we implicitly 
assume that 0(/(2)) refers to functions of a small complex number 2. 

Suppose that g(z) is a function given by an infinite power series 


g(z) = ^a k z k 

k> 0 

that converges for 2 = 20 . Then the sum of absolute values J 2 k >o\ a k zk \ also 
converges whenever \z\ < \z 0 \. If z 0 ^ 0 , we can therefore always write 

g(z) = a 0 + a 1 z H f a m z m + 0 (z m+1 ). (11) 

For we have g(z) — a 0 + a ± z H f- a m z m + z m+ 1 (a m+1 + a m+2 z H ); we need 

only show that the parenthesized quantity is bounded when \z\ < r, for some 
positive r, and it is easy to see that |a m+ i| + |a. m +2 1 t + |a m+3 | r 2 + ■ ■ ■ is an 
upper bound whenever \z\ < r < |zq|- 

For example, the generating functions listed in Section 1 . 2.9 give us many 
important asymptotic formulas valid when 2 is sufficiently small, including 

e z = 1 + 2 + ±z 2 + • • • + ~z m + 0 (z m+1 ), (12) 

z! ml 

ln(l + 2) = 2 - -2 2 + • • ■ + z m + 0(2 m+1 ), (13) 

(1 + 2)“ = l + az+ ^z 2 + ■■■+ ^z m + 0 (z m+1 ), (14) 

r ~ In ^ - 2 + H 2 z 2 + • • • + H m z m + 0 (z m+1 ), (15) 

for all nonnegative integers m. It is important to note that the hidden constants 
M and r implied by any particular O are related to each other. For example, 
the function e z is obviously 0(1) when \z\ < r, for any fixed r, since \e z \ < e' z '% 
but there is no constant M such that \e z \ < M for all values of 2. Therefore we 
need to use larger and larger bounds M as the range r increases. 

Sometimes an asymptotic series is correct although it does not correspond 
to a convergent infinite series. For example, the basic formulas that express 
factorial powers in terms of ordinary powers, 


' = £ 


k = 0 


r 

r — k\ 




k = 0 


r? r_fc + 0(n r_m_1 ), 

n r ~ k + 0 (n r ~ m ~ 1 ), 


r 

r — k 


(16) 

(w) 
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are asymptotically valid for any real r and any fixed integer m > 0, yet the sum 

OO 

E 

k = 0 

diverges for all n. (See exercise 12 .) Of course, when r is a nonnegative integer, 
n T and n- are simply polynomials of degree r, and (17) is essentially the same 
as 1 . 2 . 6 - (44). When r is a negative integer and |n| > |r|, the infinite sum 
Sfclo [ r -k\ nr ~ k does converge to n r — l/(n-l)=^; this sum can also be written 
in the more natural form YX=a { k I^}n r ~ k , using Eq. 1.2.6-(58). 

Let us give one simple example of the concepts we have introduced so far. 
Consider the quantity y/n; as n gets large, the operation of taking an nth root 
tends to decrease the value, but it is not immediately obvious whether y/n 
decreases or increases. It turns out that y/n decreases to unity. Let us consider 
the slightly more complicated quantity n(y/n - 1 ). Now (y/n - 1) gets smaller 
as n gets bigger; what happens to n(y/n - 1)? 

This problem is easily solved by applying the formulas above. We have 

= e lnn / n = 1 + (In n/n) + 0((lnn/n) 2 ), (18) 

because In n/n -» 0 as n -»• 00; see exercises 8 and 11 . This equation proves our 
previous contention that y/n -> 1. Furthermore, it tells us that 

n(y/n — 1) = n (in n/n + 0((lnn/n) 2 )) = Inn + 0((lnn) 2 /n). (19) 

In other words, n(y/n - 1 ) is approximately equal to Inn; the difference is 
0((lnn) 2 /n), which approaches zero as n approaches infinity. 

People often abuse O-notation by assuming that it gives an exact order of 
growth; they use it as if it specifies a lower bound as well as an upper bound. For 
example, an algorithm to sort n numbers might be called inefficient “because its 
running time is 0 (n 2 ).” But a running time of 0 (n 2 ) does not necessarily imply 
that the running time is not also 0 (n). There’s another notation, Big Omega, 
for lower bounds: The statement 

g(n) = n(f(n)) ( 20 ) 

means that there are positive constants L and n 0 such that 

\g(n)\> L\f(n)\ for all n>riQ. 

Using this notation we can correctly conclude that a sorting algorithm whose 
running time is fl(n 2 ) will not be as efficient as one whose running time is 
0 (n log n), if n is large enough. However, without knowing the constant factors 
implied by O and H, we cannot say anything about how large n must be before 
the 0 {n log n) method will begin to win. 

Finally, if we want to state an exact order of growth without being precise 
about constant factors, we can use Big Theta: 

g(n) = 0(/(n)) g(n) = 0 (f(n)) and g(n) = H(/(n)). (21) 


1/2 

1/2 - k 


, 1 / 2 — k 
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EXERCISES 

1. [HM01] What is liiiin-joc 0(?i _1/,:3 )? 

► 2. [ M10 } Mr. B. C. Dull obtained astonishing results by using the “self-evident” 
formula 0(/(n)) — 0(/(n)) = 0. What was his mistake, and what should the right- 
hand side of his formula have been? 

3. [M15] Multiply (Inn + 7 + 0(l/n)) by (n + 0( v / n)), and express your answer in 
O-notation. 

► 4. [ M15 ] Give an asymptotic expansion of n(^/a - 1 ), if a > 0, to terms 0(l/n 3 ). 

5. [M20] Prove or disprove: 0(/(n) + g(n)) = f(n) + 0(g(n)), if f(n) and g(n) are 
positive for all n. (Compare with ( 10 ).) 

► 6 . [M20] What is wrong with the following argument? “Since n = O(n), and 2 n = 
O(n), . . . , we have 

n n 

J2kn = J20(n) = 0(n 2 )” 
k= 1 k= 1 

7. [HM15] Prove that if m is any integer, there is no M such that e x < Mx m for 
arbitrarily large values of x. 

8 . [HM20] Prove that as n — >• 00 , (In n) m /n -» 0. 

9. [HM20] Show that e °P m ) = 1 + 0(z m ), for all fixed m > 0. 

10 . [ HM22 ] Make a statement similar to that in exercise 9 about ln(l + 0(z m )). 

► 11. [Mil] Explain why Eq. ( 18 ) is true. 

12. [HM25] Prove that n ~ k does not approach zero as k — » 00 for any integer n, 

using the fact that = (-\) h [z k ] ( ze z /(e z - 1 )) 1/2 . 

► 13. [M10] Prove or disprove: g(n) = Q(/(n)) if and only if f(n) = 0{g(n)). 

*1.2.11.2. Euler’s summation formula. One of the most useful ways to obtain 
good approximations to a sum is an approach due to Leonhard Euler. His method 
approximates a finite sum by an integral, and gives us a means to get better 
and better approximations in many cases. [Commentarii Academiae Scientiarum 
Petropolitanae 6 (1732), 68 -97.] 



Fig. 12 . Comparing a sum with an integral. 


Figure 12 shows a comparison of f" f(x) dx and YJk=\ f( k ), when n - 7. 
Euler s strategy leads to a useful formula for the difference between these two 
quantities, assuming that f(x) is a differentiable function. 
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For convenience we shall use the notation 

{x} = x mod 1 ~ x — \^x\. (i) 

Our derivation starts with the following identity: 

/■fc+l rk+1 

/ {{ x }~k)f'( x ) dx = {x-k- ±)f(x)\ k +1 - / f(x)dx 

Jk Jk 

rk -\- 1 

— \ (/( fc + 1 ) + /(*))“ / f(x)dx. ( 2 ) 

V /t 

(This follows from integration by parts.) Adding both sides of this equation for 
1 < k < n, we find that 

(M -\)f'{x)dx= ]T /(*) + !(/(«) -/(I))- I"" f(x)dx; 

l<k<n ■'! 

that is, 

/»n 

f(x)dx- i(/(n)-/(l)) + / ( 3 ) 

1N»V» ' 1 



where Si(x) is the polynomial x - |. This is the desired connection between the 
sum and the integral. 

The approximation can be carried further if we continue to integrate by 
parts. Before doing this, however, we shall discuss the Bernoulli numbers , which 
are the coefficients in the following infinite series: 


2 


e z - 1 


Bq + B\z + 


B 2 z 2 

2 ! 


+ ••• 


£ 


B k z k 

k\ 


(4) 


The coefficients of this series, which occur in a wide variety of problems, were 
introduced to European mathematicians in James Bernoulli’s Ars Conjectandi. 
published posthumously in 1713. Curiously, they were also discovered at about 
the same time by Takakazu Seki in Japan — and first published in 1712, shortly 
after his death. [See Takakazu Seki’s Collected Works (Osaka: 1974), 39-42.] 
We have 


£ 0 = 1, B i = -I, B 2 = l £3 = 0, £4 = -^; (5) 

further values appear in Appendix A. Since 

z I * _ * e * + 1 _ * e~ z + l 

e z - 1 2 2 e 2 - 1 2 e~ z - l 

is an even function, we see that 


£3 — £5 — £7 = £ g = • ■ • = 0. 


( 6 ) 
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If we multiply both sides of the defining equation (4) by e z 
coefficients of equal powers of 2, we obtain the formula 



Bk — B n + 


5nl- 


(See exercise 1 .) We now define the Bernoulli polynomial 

B m (x) = Y,(™) B kX m - k . 


1, and equate 


( 7 ) 


( 8 ) 


If 777, = 1 , then Bi{x) = B 0 x + B 1 = x — corresponding to the polynomial 
used above in Eq. (3). If m > 1 , we have B m ( 1 ) = B rn — B m ( 0 ), by (7); in other 
words, B m ({x }) has no discontinuities at integer points x. 

The relevance of Bernoulli polynomials and Bernoulli numbers to our prob- 
lem will soon be clear. We find by differentiating Eq. (8) that 


B 'm( X ) ~ k ) BkXm k 1 

k 

k 

— mB m -i(x), (9) 


and therefore when m > 1, we can integrate by parts as follows: 


1 

ml 


B m ({x})f (m \x)dx = 


1 


(to + 1)! 


(B m+ 1 (l)f( m \n) - B m+ 1 ( 0 )f^(l)) 


1 


(to + 1 )! / B ^+i(M)f im+1) ( x ) dx - 

From this result we can continue to improve the approximation, Eq. (3), and we 
obtain Euler’s general formula: 


l<fc<n 


J 2 f( k )= [ f( x ) dx ~ -/(!)) + -/'(!)) + "• 

+ ( ~ (/ (m ~ X) (n) ~ / (m - 1 >(l)) + 7? m „ 

/ n rn o 

f{x)dx + - / (fe_1) (l)) + f? mn , (10) 


using (6), where 

/ 1 \m+l rn 

Rrnn = l /, / (®) (ll) 

777 -. Ji 

The remainder R m n will be small when B m ({a:})/^ ?7l ^(a:)/TO! is very small, and 
in fact, one can show that 


Brnjjx}) 

ml 


\ B m\ 4 

~ ml ( 27 r) m 


(12) 
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when m is even. [See CMath, §9.5.] On the other hand, it usually turns out that 
the magnitude of f {m \x) gets large as m increases, so there is a “best” value of 
Tn at which |i? mn | has its least value when n is given. 

It is known that, when m is even, there is a number 9 such that 

°<*< 1. (13) 

provided that f^ m+ 2 \x) f( m +V(x) > 0 for 1 < x < n. So in these circumstances 
the remainder has the same sign as, and is less than, the first discarded term. A 
simpler version of this result is proved in exercise 3. 

Let us now apply Euler’s formula to some important examples. First, we 
set f(x) = l/x. The derivatives are f^ m \x) = (-1 ) m rn\/x m+1 , so we have’, by 
Eq. (xo), 

m „ 

(.4) 

Now we find 

m 

7 = - Inn) = £ + Um R mn . ( 15 ) 

fc= l 

The fact that li R mn = — J) B m ({x}) dx/x m+1 exists proves that the 
constant 7 does in fact exist. We can therefore put Eqs. (14) and (15) together, 
to deduce a general approximation for the harmonic numbers: 


m 

H„- 1 — In n + 7 + ^ 
k = 1 


kn k J n 


B m {{x}) dx 

j.m+\ 


m — 1 

= In n + 7 + ^ 
k= 1 


(-1 ) fc ~ 1 i? fc 

kn k 



Replacing m by m + 1 yields 


m 

H n - 1 = In n + 7 + 

k —1 


kn k 



(16) 


Furthermore, by Eq. (13) we see that the error is less than the first term 
discarded. As a particular case we have (adding 1/n to both sides) 


H n = In n + 7 + 

2 n 


1 

12n 2 


1 

120n 4 _ C ’ 


0<e< ^ 


1 

252n 6 ‘ 


This is Eq. 1.2.7-(3). The Bernoulli numbers B k for large k get very large 
(approximately (-l) 1+fc / 2 2(/d/(27r) fc ) when k is even), so Eq. (16) cannot be 
extended to a convergent infinite series for any fixed value of n. 
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The same technique can be applied to deduce Stirling’s approximation. This 
time we set f(x ) = lnx, and Eq. ( 10 ) yields 

B k {- l) fc 


ln(n — 1)! = nlnn — n + 1 — | Inn + ^ 

1 <k<m 

Proceeding as above, we find that the limit 


k(k - 1) 


( n fc - 1 *) + Rmn ' ( 1 7 ) 


lim (Inn! - n Inn + n - \ Inn) = 1 + h lim R rnn 

n^oo z ' 111 1 x 


1 <k<m 


k(k — 1) n—>oo 


exists; let it be called a (“Stirling’s constant”) temporarily. We get Stirling’s 
result 


Inn! = (n + |) Inn — n + a + 


1 <k<m 


B k {- l) fc 

k(k — 1 )n k ~ l 


+ o 


( 18 ) 


In particular, let m = 5; we have 

1 „„! = (» + 1) Inn - » + " + ^ + oQ). 

Now we can take the exponential of both sides: 


n! = e“ s/n I — ) exp 


1 


12n 360n 3 


+ o - 


(*))■ 


Using the fact that e a — \J2 tx (see exercise 5), and expanding the exponential, 
we get our final result: 


n! = V27m| — 

, e 


) fl + — + — 

) \ 12n 288 


139 


571 


n 2 51840n 3 2488320n 4 


+ °b • ( x 9) 


EXERCISES 

1. [Ml 8] Prove Eq. ( 7 ). 

2. [HM20] Note that Eq. ( 9 ) follows from Eq. ( 8 ) for any sequence B„ , not only for 
the sequence defined by Eq. ( 4 ). Explain why the latter sequence is necessary for the 
validity of Eq. ( 10 ). 

3. [HM20] Let C mn = (B m / m!)(/ (m- P (n) — / l ' m_ 1 ^(l)) be the mth correction term 

in Euler’s summation formula. Assuming that has a constant sign for all x in 

the range 1 < x < n, prove that |_R mn | < |C mn | when m = 2k > 0; in other words, 
show that the remainder is not larger in absolute value than the last term computed. 

► 4. [HM20] ( Sums of powers.) When f(x) = x m , the high-order derivatives of / are 
all zero, so Euler’s summation formula gives an exact value for the sum 

Sm(n)= J2 fcm 

0 </e<n 

in terms of Bernoulli numbers. (It was the study of S m (n) for m = 1, 2, 3, . . . that 
led Bernoulli and Seki to discover those numbers in the first place.) Express S m (n) in 
terms of Bernoulli polynomials. Check your answer for m = 0, 1, and 2. (Note that 
the desired sum is performed for 0 < k < n instead of 1 < k < n; Euler’s summation 
formula may be applied with 0 replacing 1 throughout.) 
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[HM30] Given that 


■^(-) n (i + o(~) 

\ej \ \nj 


show that K = \/2n by using Wallis’s product (exercise 1.2.5-18). [Hint: Consider ( 2 ^) 
for large values of n.] 

► 6. [HM30] Show that Stirling’s approximation holds for noninteger n as well: 


r(x + i) = ' / 2^(f) X (^i + 0 (^)^t 


x > a > 0. 


[Hint: Let /( x) = ln(a; + c) in Euler’s summation formula, and apply the definition of 
F(a:) given in Section 1.2.5.] 

► 7. [HM32] What is the approximate value of 1 1 2 2 3 3 . .. u n ? 

8. [M23] Find the asymptotic value of In (an 2 +bn)\ with absolute error 0(rU 2 ). Use 
it to compute the asymptotic value of ( c ” 2 )/c n (” rl 2 ) with relative error 0(n -2 ), when c 
is a positive constant. Here absolute error e means that (truth) = (approximation) +e; 
relative error e means that (truth) = (approximation) (1 + e). 

► 9. [M25] Find the asymptotic value of ( 2 ^) with a relative error of 0(n~ 3 ), in two 
ways: (a) via Stirling’s approximation; (b) via exercise 1.2.6-47 and Eq. 1.2.11.1-(i6). 

*1.2.11.3. Some asymptotic calculations. In this subsection we shall inves- 
tigate the following three intriguing sums, in order to deduce their approximate 
values: 


P(n) = 1 + 


n — 2 n — 2 


(n — k) k (n — k)\ 

71 ! 


. n — 1 n-ln-2 

Q(n) = 1 + + + 

n n n 


E Tl: 

(n — k) \ n k ’ 

k = l v ’ 


R(n) = 1 + 


n + 1 n + 2 


(n + fc)! 


These functions, which are similar in appearance yet intrinsically different, arise 
in several algorithms that we shall encounter later. Both P(n) and Q(n) are 
finite sums, while R(n) is an infinite sum. It seems that when n is large, all 
three sums will be nearly equal, although it is not obvious what the approximate 
value of any of them will be. Our quest for approximate values of these functions 
will lead us through a number of very instructive side results. (You may wish to 
stop reading temporarily and try your hand at studying these functions before 
going on to see how they are attacked here.) 

First, we observe an important connection between Q(n) and R(n ): 


Q{n ) + R(n) = 


1 + n + • • • + 


(n - 1)! 


(n + 1)! 


( 4 ) 
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Stirling’s formula tells us that n! e"/n n is approximately \/ 2 ? m, so we can guess 
that Q(n) and R(n) will each turn out to be roughly equal to \J nn/ 2 . 

To get any further we must consider the partial sums of the series for e n . 
By using Taylor’s formula with remainder, 

f(n) ("n\ T n px j.n 

f( x ) = /(0) + f'(ty x + • ' • + V +/ — - J (n+1) (2; — t) (it, (5) 

n! J 0 n\ 

we are soon led to an important function known as the incomplete gamma 
function: 

7(0,2:) = f e~ t t a ~ 1 dt. (6) 

Jo 

We shall assume that a > 0 . By exercise 1 . 2 . 5 - 20 , we have 7(0,00) = T(a); this 
accounts for the name “incomplete gamma function.” It has two useful series 
expansions in powers of x (see exercises 2 and 3 ): 


7 {a,x) 


a 


1 2 
o+l + 2! (a + 2) 


r , , x x 

e 7 (a, x) — 1- 


,0. — 1 


r a+2 


a a(a+l) a(a+l)(a + 2) 


(_1 ^k x k+a 

f-' k\(k + a ) ’ 


E 


x k+a 

a(a + 1) . . . (a + k) 


( 7 ) 

( 8 ) 


From the second formula we see the connection with R(n): 


R(n) 


n\ e n f 7 (n, n) \ 
n n \ (n — 1 )! ) 


(9) 


This equation has purposely been written in a more complicated form than 
necessary, since 7 (n,n) is a fraction of 7(n, 00) = T(n) = (n — 1)!, and n\e n /n n 
is the quantity in (4). 

The problem boils down to getting good estimates of 7 (n,n)/(n — 1 )!. We 
shall now determine the approximate value of 7(2: + 1 , x + y)/Y(x + 1 ), when y 
is fixed and x is large. The methods to be used here are more important than 
the results, so the reader should study the following derivation carefully. 

By definition, we have 


7 ( 2 - + 1, x + y) 

r(x + 1) 


rx+y 


r(® + 1) Jo 

= 1 - 


-H x dt 


1 poo 1 px+y 

TU TTT / e - t t x dt +— — -/ e-H x dt. ( 10 ) 

F(x + l)J x T{x+l)J x 


Let us set 


h = 

1 2 = 


nOO 

L 1 


e ~ l t x dt, 
x+y 

e- l t x dt. 


and consider each integral in turn. 
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Estimate of I r : We convert I x to an integral from 0 to infinity by substituting 
t~ x(l + u); we further substitute v = u — ln(l + u), dv = (l - 1/(1 + u)) du , 
which is legitimate since v is a monotone function of u: 


+ (11 ) 


/i = e ^ / are _<ru (l + u) x du = e~ x x x 

Jo 

In the last integral we will replace 1 + 1/u by a power series in v. We have 

\u 4 


! " 2 ' u - 4 X + -" = («72)(l-Itt + §u 2 -|« 3 + -..). 


V = — |u 


1331 


u 5 + 0 (u 6 ). 


4“ 5 

Setting in = s/2v, we have therefore 

u> = u(l-|u+iu 2 -!u 3 + ---) 1/2 =u-iu 2 + ^u 3 -^u 4 + 129g5 

(This expansion may be obtained by the binomial theorem; efficient methods for 
performing such transformations, and for doing the other power series manipu- 
lations needed below, are considered in Section 4.7.) We can now solve for u as 
a power series in w: 


1 2 1 3 

u — w -\ — w H ur — 

3 36 


^ + o(«> 6 y, 


1 + - = 1 
u 


1 

w 


1 


270 

2 


4320 

1 


U W ~U5 W + ^ W ' + ° {W ‘ ] 


864 


- 1/2 

— -y 


s/2 


l + ^ 

3 12 


4 V2 

v H v 

135 432 


3/2 


+ 0{v 2 ). ( 12 ) 


In all of these formulas, the O-notation refers to small values of the argu- 
ment, that is, |u| < r, \v\ < r, |w| < r for sufficiently small positive r. Is this 
good enough? The substitution of 1 + 1/u in terms of v in Eq. ( 11 ) is supposed 
to be valid for 0 < v < 00 , not only for |v| < r. Fortunately, it turns out that 
the value of the integral from 0 to 00 depends almost entirely on the values of 
the integrand near zero. In fact, we have (see exercise 4) 



(l + ^)du = 0(e— ) 


(13) 


for any fixed r > 0 and for large x. We are interested in an approximation up 
to terms 0(x“ m ), and since 0((l/e r ) x ) is much smaller than 0(x ~ m ) for any 
positive r and m, we need integrate only from 0 to r, for any fixed positive r. We 
therefore take r to be small enough so that all the power series manipulations 
done above are justified (see Eqs. 1.2.11.1-(n) and 1. 2. 11. 3 -/ 13 )) . 

Now 


f 

Jo 


u v a dv = 


1 r 

X a Jo 


° q q a dq = — T(q + 1 ), if a > - 1 ; 


so by plugging the series ( 12 ) into the integral ( 11 ) we have finally 


h = e- x x* 


V/ 2 + - + 


\Z2n 
~24~ 3 


-1/2. 


\/2tt 


135 X + '576 


.-3/2 _ 


- 0(x 


(14) 


(15) 


1.2.11.3 


SOME ASYMPTOTIC CALCULATIONS 


119 


Estimate of I 2 : In the integral I 2 , we substitute t — u + x and obtain 
h = e~ x x x 


u\ x 

1 H — ) du. 
x ) 


(16) 


Now 


“( 1 + 3 =exp(- U + xl„( 1 + ^))=exp( 1 ^ + 


3z 2 


0( a 


u ‘ 
2x 


U 4 U 3 _q 

8 ^ + 3^ +0(:C } 


for 0 < u < y and large x. Therefore we find that 


I 2 = e x x x ( y - ~x 




17) 


Finally, we analyze the coefficient e~ x x x /T(x + 1) that appears when we 
multiply Eqs. (15) and (17) by the factor l/r(i + 1) in (10). By Stirling’s 
approximation, which is valid for the gamma function by exercise 1.2.11.2-6, we 
have 


r(x + 1) 


-l/12x + 0(x -3 ) 
'J'l'KX 

r-Va _ _ 


-3/2 


+ 


: X~ 5/2 + 0{X~ 7/2 ). (18) 


V2n 12\/27r ~ ' 288^" 

And now the grand summing up: Equations (10), (15), (17), and (18) yield 
Theorem A. For large values of x, and fixed y, 


7(2:+ 1 ,x + y) 

r(x + i) 


y ^ Ul \ x - 


1/2 


+ 


1 ( 23 


y y 


V270 12 6 

+ 0(x~ 5 / 2 ). | 


-3/2 


(19) 


The method we have used shows how this approximation could be extended to 
further powers of x as far as we please. 

Theorem A can be used to obtain the approximate values of R(n) and Q(n), 
by using Eqs. (4) and (9), but we shall defer that calculation until later. Let us 
now turn to P(n), for which somewhat different methods seem to be required. 
We have 


" k- k k\ 

P{n) = 2 ^ 




71! 


f E k n+ 1 / 2 e~ k (l + ^ + 0 (fc~ 2 )) . (20) 


fc= 0 k = 0 

Thus to get the values of P(n), we must study sums of the form 


*3 = 0 


n+l/2 e ~ k 
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Let /( x) — x n+1 / 2 e x and apply Euler’s summation formula: 

n 

J2k n+1/2 e~ k = 

k - 0 


x n+ 1/2 e -* dj . + l n n+l/2 e -n + 


— R. ( 21 ) 


A crude analysis of the remainder (see exercise 5) shows that R = 0(n n e ~ n ); 
and since the integral is an incomplete gamma function, we have 

n 

J2 k n+1/2 e~ k = 7(71 + §, n) + \r i n+1 ' 2 e~ n + 0{n n e~ n ). ( 22 ) 

k = 0 


Our formula, Eq. ( 20 ), also requires an estimate of the sum 

n 

J2k n ~ 1/2 e~ k = fc (n ~ 1)+1/2 e~ fc + n”- 1 / 2 e- n , 

k = 0 0<fc<n-l 


and this can also be obtained by Eq. ( 22 ). 

We now have enough formulas at our disposal to determine the approximate 
values of P(n ), Q(n), and R{n), and it is only a matter of substituting and 
multiplying, etc. In this process we shall have occasion to use the expansion 

(» + «) n+/? = " n+/3 e“ (i + a (/? - | ^ + 0(n~ 2 )^j , ( 23 ) 

which is proved in exercise 6 . The method of ( 21 ) yields only the first two terms 
in the asymptotic series for P(n ); further terms can be obtained by using the 
instructive technique described in exercise 14. 

The result of all these calculations gives us the desired asymptotic formulas: 


Pin) 

Q(n) 

R{n) 



V 2 + 3 + 12 V 2n + 135n 


irk/ 2 ^ +0(n “ 2) ’ 
288 \[^ + °^ n 2 ^ 
^8\[^P + ° (n ~ 2) - 


(24) 

(25) 

( 26 ) 


The functions studied here have received only light treatment in the pub- 
lished literature. The first term \J iru f 2 in the expansion of P{n ) was given by 
H. B. Dernuth [Ph.D. thesis (Stanford University, October 1956), 67-68]. Using 
this result, a table of P[n ) for n < 2000, and a good slide rule, the author 
proceeded in 1963 to deduce the empirical estimate P(n) k yj nn/2 - 0.6667 + 
0.575/ yjn. It was natural to conjecture that 0.6667 was really an approximation 
to 2/3, and that 0.575 would perhaps turn out to be an approximation to 
7 = 0.57721... (why not be optimistic?). Later, as this section was being 
written, the correct expansion of P(n) was developed, and the conjecture 2/3 
was verified; for the other coefficient 0.575 we have not 7 but ~ \Jn/2 « 0.5744. 
This nicely confirms both the theory and the empirical estimates. 
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Formulas equivalent to the asymptotic values of Q(n ) and R(n) were first 
determined by the brilliant self-taught Indian mathematician S. Ramanujan, 
who posed the problem of estimating n\e n /2n n — Q(n) in J. Indian Math. Soc. 
3 (1911), 128; 4 (1912), 151 152. In his answer to the problem, he gave the 

asymptotic series | + yf ~ 8505 n_3 ^ » which goes considerably 

beyond Eq. (25). His derivation was somewhat more elegant than the method 
described above; to estimate Ji, he substituted t = x + u\/2x , and expressed 
the integrand as a sum of terms of the form Cjk /[(* 3 exp(— u 2 )u?x k / 2 du. The 
integral I 2 can be avoided completely, since ay (a, as) = x a e~ x +7(0+ 1, x ) when 
a > 0; see (8). An even simpler approach to the asymptotics of Q(n), perhaps 
the simplest possible, appears in exercise 20. The derivation we have used, 
which is instructive in spite of its unnecessary complications, is due to R. Furch 
[ Zeitschrift fur Physik 112 (1939), 92-95], who was primarily interested in the 
value of y that makes 7(2: + 1, x + y) — T(x + l)/2. The asymptotic properties 
of the incomplete gamma function were later extended to complex arguments by 
F. G. Tricomi [Math. Zeitschrift 53 (1950), 136-148], See also N. M. Temme, 
Math. Comp. 29 (1975), 1109-1114; SIAM J. Math. Anal. 10 (1979), 757-766. 
H. W. Gould has listed references to several other investigations of Q(n) in AMM 
75 (1968), 1019-1021. 

Our derivations of the asymptotic series for P{n), Q(n), and R{n) use only 
simple techniques of elementary calculus; notice that we have used different 
methods for each function! Actually we could have solved all three problems 
using the techniques of exercise 14, which are explained further in Sections 5.1.4 
and 5.2.2. That would have been more elegant but less instructive. 

For additional information, interested readers should consult the beautiful 
book Asymptotic Methods in Analysis by N. G. de Bruijn (Amsterdam: North- 
Holland, 1958). See also the more recent survey by A. M. Odlyzko [Handbook 
of Combinatorics 2 (MIT Press, 1995), 1063-1229], which includes 65 detailed 
examples and an extensive bibliography. 

EXERCISES 

1. [ HM20 } Prove Eq. ( 5 ) by induction on n. 

2. [HM20] Obtain Eq. ( 7 ) from Eq. ( 6 ). 

3. [M20] Derive Eq. ( 8 ) from Eq. ( 7 ). 

► 4. [HM10] Prove Eq. ( 13 ). 

5. [HM24] Show that R in Eq. ( 21 ) is 0(n n e~ n ). 

► 6 . [HM20] Prove Eq. ( 23 ). 

► 7. [HM30] In the evaluation of I 2 , we had to consider J e “ |l 1 — du. Give an 

asymptotic representation of 0 

to terms of order 0(x ~ 2 ), when y is fixed and x is large. 
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8 . [HM30] If f(x) = 0(x r ) as x — > oo and 0 < r < 1 , show that 


f ,(x) u\ x f f(x > ( -v? u 3 (-l) m- 1 u m \ 

/ e I 1 + - du= / exp — h — — 1 - 1 — ) du + 0(x~ s ) 

Jo V x) J 0 \ 2x 3x 2 mx m_1 J ' y 

if m = f(s + 2r)/(l - r)] . [This proves in particular a result due to Tricomi: If 
f(x) = 0(y/x ), then 


rw 


1 H — ) du = 
x ) 


V2x f 

J o 


f(x)/V 2x 


e 1 dt + 0 ( 1 ).] 


► 9. [HM36] What is the behavior of y(x + 1, px)/T(x + 1 ) for large xJ (Here p is a 
real constant; and if p < 0 , we assume that x is an integer, so that t x is well defined for 
negative t.) Obtain at least two terms of the asymptotic expansion, before resorting 
to O-terms. 

10 . [ HM34 ] Under the assumptions of the preceding problem, with p / 1, obtain the 
asymptotic expansion of y(x + 1 , px + py/(p - 1 )) - j(x + 1 , px), for fixed y, to terms 
of the same order as obtained in the previous exercise. 

► 11 . [HM35] Let us generalize the functions Q(n) and R(n ) by introducing a 
ter x : 

t \ i , n— 1 , n — 1 n — 2 2 

Qx(n) = 1 H — x H x + ■ 


. parame- 


Rx{n) — 1 H - 


n 

n 


-x + 


n 

n 


n 

n 


2 i 

■x + ■ 


n -\- 1 n + ln + 2 ’ 

Explore this situation and find asymptotic formulas when x ^ 1. 

12 . [HM20] The function / Q x e“* /2 dt that appeared in connection with the normal 
distribution (see Section 1.2.10) can be expressed as a special case of the incomplete 
gamma function. Find values of a, b, and y such that 67 ( 0 , y) equals f* e~ f2/2 dt. 

13 . [ HM42 ] (S. Ramanujan.) Prove that R(n) - Q(n) = § + 8/(l35(n + 6(n))), 
where < 0(n) < (This implies the much weaker result R(n + 1) - Q(n + 1 ) < 
R(n) - Q(n).) 

14 . [HM39] (N. G. de Bruijn.) The purpose of this exercise is to find the asymptotic 
expansion of Ylk=o k n+a e~ k for fixed a,asB-> 00 . 

a) Replacing k by n-k, show that the given sum equals n n+a e~ n J2k=o e _fc 2 / 2 n /(fc, n), 
where 

k 3 k 4 


/(fc, n) = (l — ex p (- 


3n 2 4n 3 




b) Show that for all m > 0 and t > 0, the quantity f(k, n) can be written in the form 


E 


c% 7 k 


2 *+j -*-j 


+ 0 (n (m+1,( - 1/2+3e )), if 0 < k < n 1/2+e . 


0<i<j <7i 


c) Prove that as a consequence of (b), we have 

E e ~ k2/2n f(k, n)= ^ c ij n- i ~ i Y k 2i+3 e~ k2/2n + 0( n - m/2+s ), 

k=0 0<i<j<m k> 0 

for all 8 > 0. [Hint: The sums over the range n 1/,2+e < k < 00 are 0(n~ r ) for 
all r.] 
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d) Show that the asymptotic expansion of ^ Zk>o k f e~ k ' /2n for fixed t > 0 can be 
obtained by Euler’s summation formula. 

e) Finally therefore 


Y,k n+a e- k 

k = 0 



- a + 


a 


, 1 1 2 

H — a H — a 
12 2 2 


+ 0(n 


->) 


this computation can in principle be extended to 0(n r ) for any desired r. 

15. [HM20] Show that the following integral is related to Q(n): 

r( i+ 3' e ~‘ dz - 

16. [M24] Prove the identity 

= (— l) n ( n - 1)!, when n > 0. 
k 

17. [HM29] (K. W. Miller.) Symmetry demands that we consider also a fourth series, 
which is to P(n ) as R(n ) is to Q(n): 

ql \ . . n n n+ 1 V- (n + fc-1)! 

71 n+1 n + 2n + 2 ^ (n — 1)! (n + k) k 


What is the asymptotic behavior of this function? 

18. [M25] Show that the sums (£)fc fc (n — k) n ~ k and (^)(fc + l) fc (n — can 

be expressed very simply in terms of the Q function. 

19. [HM30] (Watson’s lemma.) Show that if the integral C n = J 0 °° e~ nx f(x) dx exists 
for all large n, and if f(x) = 0(x a ) for 0 < x < r, where r > 0 and a > -1, then 
C n =0(n- l ~ a ). 

► 20. [HM30] Let u = w + ^ w 3 - ^w A 4 = J2T=i c * be the P ower series 

solution to the equation w = ( u 2 — |u 3 + | u 4 — |u 5 + • • • ) 1 ^ 2 , as in ( 12 ). Show that 


m —1 \ 1 — fc /2 

Q(n) + 1= kc k T(k/2)[^j +0(n 1 ~ m/2 ) 

k = 1 

for all m > 1. [Hint: Apply Watson’s lemma to the identity of exercise 15.] 


I feel as if I should succeed in doing something in mathematics, 
although I cannot see why it is so very important. 

— HELEN KELLER (1898) 
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1.3. MIX 

In many places throughout this book we will have occasion to refer to a com- 
puter’s internal machine language. The machine we use is a mythical computer 
called “MIX.” MIX is very much like nearly every computer of the 1960s and 
1970s, except that it is, perhaps, nicer. The language of MIX has been designed 
to be powerful enough to allow brief programs to be written for most algorithms, 
yet simple enough so that its operations are easily learned. 

The reader is urged to study this section carefully, since MIX language 
appears in so many parts of this book. There should be no hesitation about 
learning a machine language; indeed, the author once found it not uncommon to 
be writing programs in a half dozen different machine languages during the same 
week! Everyone with more than a casual interest in computers will probably get 
to know at least one machine language sooner or later. MIX lias been specially 
designed to preserve the simplest aspects of historic computers, so that its 
characteristics are easy to assimilate. 

/^\ However, it must be admitted that MIX is now quite obsolete. Therefore 
JL MIX will be replaced in subsequent editions of this book by a new machine 
called MMIX, the 2009. MMIX will be a so-called reduced instruction set computer 
(RISC), which will do arithmetic on 64-bit words. It will be even nicer than MIX. 
and it will be similar to machines that have become dominant during the 1990s. 

The task of converting everything in this book from MIX to MMIX will take a 
long time; volunteers are solicited to help with that conversion process. Mean- 
while, the author hopes that people will be content to live for a few more years 
with the old-fashioned MIX architecture — which is still worth knowing, because 
it helps to provide a context for subsequent developments. 

1.3.1. Description of MIX 

MIX is the world’s first polyunsaturated computer. Like most machines, it has 
an identifying number — the 1009. This number was found by taking 16 actual 
computers very similar to MIX and on which MIX could easily be simulated, then 
averaging their numbers with equal weight: 

[(360 + 650 + 709 + 7070 + U3 + SS80 + 1107 + 1604 + G20 + B220 

+ S2000 + 920 + 601 + H800 + PDP-4 + II)/16j = 1009. (i) 

The same number may also be obtained in a simpler way by taking Roman 
numerals. 

MIX has a peculiar property in that it is both binary and decimal at the same 
time. MIX programmers don't actually know whether they are programming a 
machine with base 2 or base 10 arithmetic. Therefore algorithms written in 
MIX can be used on either type of machine with little change, and MIX can be 
simulated easily on either type of machine. Programmers who are accustomed 
to a binary machine can think of MIX as binary; those accustomed to decimal 
may regard MIX as decimal. Programmers from another planet might choose to 
think of MIX as a ternary computer. 
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Words. The basic unit of MIX data is a byte. Each byte contains an unspecified 
amount of information, but it must be capable of holding at least 64 distinct 
values. That is, we know that any number between 0 and 63, inclusive, can be 
contained in one byte. Furthermore, each byte contains at most 100 distinct 
values. On a binary computer a byte must therefore be composed of six bits; on 
a decimal computer we have two digits per byte.* 

Programs expressed in Mix's language should be written so that no more 
than sixty-four values are ever assumed for a byte. If we wish to treat the 
number 80, we should always leave two adjacent bytes for expressing it, even 
though one byte is sufficient on a decimal computer. An algorithm in MIX should 
work properly regardless of how big a byte is. Although it is quite possible to 
write programs that depend on the byte size, such actions are anathema to the 
spirit of this book; the only legitimate programs are those that would give correct 
results with all byte sizes. It is usually not hard to abide by this ground rule, 
and we will thereby find that programming a decimal computer isn’t so different 
from programming a binary one after all. 

Two adjacent bytes can express the numbers 0 through 4,095. 

Three adjacent bytes can express the numbers 0 through 262,143. 

Four adjacent bytes can express the numbers 0 through 16,777,215. 

Five adjacent bytes can express the numbers 0 through 1,073,741,823. 

A computer word consists of five bytes and a sign. The sign portion has 
only two possible values, + and — . 

Registers. There are nine registers in MIX (see Fig. 13): 

The A-register (Accumulator) consists of five bytes and a sign. 

The X-register (Extension), likewise, comprises five bytes and a sign. 

The I-registers (Index registers) II, 12, 13, 14, 15, and 16 each hold two bytes 
together with a sign. 

The J-register (Jump address) holds two bytes; it behaves as if its sign is 
always +. 

We shall use a small letter “r”, prefixed to the name, to identify a MIX register. 
Thus, “rA” means “register A.” 

The A-register has many uses, especially for arithmetic and for operating 
on data. The X-register is an extension on the “right-hand side” of rA, and it 
is used in connection with rA to hold ten bytes of a product or dividend, or 
it can be used to hold information shifted to the right out of rA. The index 
registers ill , rI2, rI3, rI4, rI5, and rI6 are used primarily for counting and for 
referencing variable memory addresses. The J-register always holds the address 
of the instruction following the most recent “jump” operation, and it is primarily 
used in connection with subroutines. 

* Since 1975 or so, the word “byte” has come to mean a sequence of precisely eight binary 
digits, capable of representing the numbers 0 to 255. Real-world bytes are therefore larger than 
the bytes of the hypothetical MIX machine; indeed, Mix’s old-style bytes are just barely bigger 
than nybbles. When we speak of bytes in connection with MIX we shall confine ourselves to the 
former sense of the word, harking back to the days when bytes were not yet standardized. 
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Fig. 13. The MIX computer. 


Besides its registers, MIX contains 

an overflow toggle (a single bit that is either “on” or “off”); 
a comparison indicator (having three values: LESS, EQUAL, or GREATER); 
memory (4000 words of storage, each word with five bytes and a sign); 
and input-output devices (cards, tapes, disks, etc.). 

Partial fields of words. The five bytes and sign of a computer word are 
numbered as follows: 


0 

1 

2 

3 

4 

5 

± 

Byte 

Byte 

Byte 

Byte 

Byte 


(2) 
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Most of the instructions allow a programmer to use only part of a word if desired. 
In such cases a nonstandard “field specification” can be given. The allowable 
fields are those that are adjacent in a computer word, and they are represented 
by (L:R), where L is the number of the left-hand part and R is the number of 
the right-hand part of the field. Examples of field specifications are: 

(0:0), the sign only. 

(0:2), the sign and the first two bytes. 

(0:5), the whole word; this is the most common field specification. 

(1:5), the whole word except for the sign. 

(4:4), the fourth byte only. 

(4:5), the two least significant bytes. 

The use of field specifications varies slightly from instruction to instruction, 
and it will be explained in detail for each instruction where it applies. Each 
field specification (L : R) is actually represented inside the machine by the single 
number 8L + R; notice that this number fits easily in one byte. 

Instruction format. Computer words used for instructions have the following 
form: . , 


0 

1 

2 

3 

4 

5 

± 

A 

A 

1 

I 

F 

C 


The rightmost byte, C, is the operation code telling what operation is to 
be performed. For example, C = 8 specifies the operation LDA, “load the A- 
regist.er.” 

The F-byte holds a modification of the operation code. It is usually a field 
specification (L:R) = 8L + R; for example, if C = 8 and F = 11, the operation 
is “load the A-register with the (1:3) field.” Sometimes F is used for other 
purposes; on input-output instructions, for example, F is the number of the 
relevant input or output unit. 

The left-hand portion of the instruction, ±AA, is the address. (Notice that 
the sign is part of the address.) The I-field, which comes next to the address, 
is the index specification, which may be used to modify the effective address. 
If I = 0, the address ±AA is used without change; otherwise I should contain 
a number i between 1 and 6, and the contents of index register I i are added 
algebraically to ±AA before the instruction is carried out; the result is used as 
the address. This indexing process takes place on every instruction. We will use 
the letter M to indicate the address after any specified indexing has occurred. 
(If the addition of the index register to the address ±AA yields a result that 
does not fit in two bytes, the value of M is undefined.) 

In most instructions, M will refer to a memory cell. The terms “memory 
cell” and “memory location” are used almost interchangeably in this book. We 
assume that there are 4000 memory cells, numbered from 0 to 3999; hence every 
memory location can be addressed with two bytes. For every instruction in which 
M refers to a memory cell we must have 0 < M < 3999, and in this case we will 
write CONTENTS (M) to denote the value stored in memory location M. 
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On certain instructions, the “address” M has another significance, and it 
may even be negative. Thus, one instruction adds M to an index register, and 
such an operation takes account of the sign of M. 

Notation. To discuss instructions in a readable manner, we will use the notation 


OP ADDRESS, 1(F) (4) 

to denote an instruction like (3). Here OP is a symbolic name given to the 
operation code (the C-part) of the instruction; ADDRESS is the ±AA portion: I 
and F represent the I- and F-fields, respectively. 

If I is zero, the “,I” is omitted. If F is the normal F-specification for this 
particular operator, the “(F)” need not be written. The normal F-specification 
for almost all operators is (0:5), representing a whole word. If a different F is 
normal, it will be mentioned explicitly when we discuss a particular operator. 

For example, the instruction to load a number into the accumulator is called 
LDA and it is operation code number 8. We have 


Conventional representation 
LDA 2000,2(0:3) 

LDA 2000,2(1:3) 

LDA 2000(1:3) 

LDA 2000 
LDA -2000,4 


Actual numeric instruction 


+ 

' 2000 

2 

3 

8 

+ 

2000 

2 

11 

8 

+ 

2000 

0 

11 

8 

+ 

2000 

0 

5 

8 

- 

2000 

4 

5 

8 


(5) 


The instruction “LDA 2000,2(0:3)” may be read “Load A with the contents of 
location 2000 indexed by 2, the zero-three field.” 

To represent the numerical contents of a MIX word, we will always use a box 
notation like that above. Notice that in the word 


+ ’ 2000 

2 

1 

3 

8 


the number +2000 is shown filling two adjacent bytes and sign; the actual 
contents of byte (1:1) and of byte (2:2) will vary from one MIX computer to 
another, since byte size is variable. As a further example of this notation for MIX 
words, the diagram 


- 10000 


3000 


represents a word with two fields, a three- byte-plus-sign field containing -10000 
and a two-byte field containing 3000. When a word is split into more than one 
field, it is said to be “packed.” 


Rules for each instruction. The remarks following (3) above have defined 
the quantities M, F, and C for every word used as an instruction. We will now 
define the actions corresponding to each instruction. 





1.3.1 


DESCRIPTION OF MIX 129 


Loading operators. 


• LDA (load A). C = 8; F = field. 

The specified field of CONTENTS (M) replaces the previous contents of register A. 

On all operations where a partial field is used as an input, the sign is used 
if it is a part of the field, otherwise the sign + is understood. The field is shifted 
over to the right-hand part of the register as it is loaded. 

Examples: If F is the normal field specification (0:5), everything in location 
M is copied into rA. If F is (1:5), the absolute value of CONTENTS(M) is loaded 
with a plus sign. If M contains an instruction word and if F is (0:2), the “±AA” 


field is loaded as 


± 

0 

0 

0 

A A 


Suppose location 2000 contains the word 


- 

80 

3 

5 

4 


( 6 ) 


then we get the following results from loading various partial fields: 

Instruction Contents of rA afterwards 
LDA 2000 
LDA 2000(1:5) 

LDA 2000(3:5) 

LDA 2000(0:3) 

LDA 2000(4:4) 

LDA 2000(0:0) 

LDA 2000(1:1) 

(The last example has a partially unknown effect, since byte size is variable.) 

• LDX (load X). C = 15; F = field. 

This is the same as LDA. except that rX is loaded instead of rA. 

• LDi (load i). C = 8 + i; F = field. 

This is the same as LDA, except that rli is loaded instead of rA. An index register 
contains only two bytes (not five) and a sign; bytes 1, 2, 3 are always assumed 
to be zero. The LDi instruction is undefined if it would result in setting bytes 1, 
2, or 3 to anything but zero. 

In the description of all instructions, “i” stands for an integer, 1 < i < 6. 
Thus, LDi stands for six different instructions: LDI, LD2, . . . , LD6. 

• LDAN (load A negative). C = 16; F = field. 

• LDXN (load X negative). C = 23; F = field. 

• LDiN (load i negative). C = 16 + i; F = field. 

These eight instructions are the same as LDA, LDX, LDi, respectively, except that 
the opposite sign is loaded. 


- 

80 

3 

5 

4 

+ 

8 

0 

3 

5 

4 

+ 

0 

0 

3 

5 

4 
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0 

0 
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0 

3 

+ 

0 

0 

0 

0 

5 
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0 

0 

0 

0 

0 

+ 

0 

0 

0 

0 

7 





130 BASIC CONCEPTS 


1.3.1 


Storing operators. 

• STA (store A). C = 24; F = field. 

A portion of the contents of rA replaces the field of CONTENTS (M) specified by F. 
The other parts of CONTENTS (M) are unchanged. 

On a store operation the field F has the opposite significance from the load 
operation: The number of bytes in the field is taken from the right-hand portion 
of the register and shifted left if necessary to be inserted in the proper field of 
CONTENTS (M). The sign is not altered unless it is part of the field. The contents 
of the register are not affected. 

Examples: Suppose that location 2000 contains 


and register A contains 


Then: 

Instruction 


- 

1 

2 

3 

4 

5 


+ 

6 

7 

8 

9 

0 


Contents of location 2000 afterwards 


STA 2000 
STA 2000(1:5) 

STA 2000(5:5) 

STA 2000(2:2) 

STA 2000(2:3) 

STA 2000(0:1) 

• STX (store X). C = 31; F = field. 

Same as STA, except that rX is stored rather than rA. 


+ 

6 

7 

8 

9 

0 

- 

6 

7 

8 

9 

0 

- 

1 

2 

3 

4 

0 

- 

1 

0 

3 

4 

5 

- 

1 

9 

0 

4 

5 

+ 

0 

2 

3 

4 
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• ST? (store i). C = 24 + i; F = field. 

Same as STA, except that rl? is stored rather than rA. Bytes 1, 2, 3 of an index 
register are zero; thus if rll contains 


± m 


it behaves as though it were 


± 

0 

0 

0 

m 

n 


• STJ (store J). C = 32; F = field. 

Same as ST?’, except that rJ is stored and its sign is always +. 

With STJ the normal held specification for F is (0:2), not (0:5). This is 
natural, since STJ is almost always done into the address field of an instruction. 

• STZ (store zero). C = 33; F = field. 

Same as STA, except that plus zero is stored. In other words, the specified field 
of CONTENTS (M) is cleared to zero. 
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Arithmetic operators. On the add, subtract, multiply, and divide operations, 
a field specification is allowed. A field specification of “(0:6)” can be used to 
indicate a “floating point” operation (see Section 4.2), but few of the programs 
we will write for MIX will use this feature, since we will primarily be concerned 
with algorithms on integers. 

The standard field specification is, as usual, (0:5). Other fields are treated 
as in LDA. We will use the letter V to indicate the specified field of CONTENTS (M) ; 
thus, V is the value that would have been loaded into register A if the operation 
code were LDA. 

• ADD. C = 1; F = field. 

V is added to rA. If the magnitude of the result is too large for register A, the 
overflow toggle is set on, and the remainder of the addition appearing in rA is as 
though a “1” had been carried into another register to the left of rA. (Otherwise 
the setting of the overflow toggle is unchanged.) If the result is zero, the sign of 
rA is unchanged. 

Example: The sequence of instructions below computes the sum of the five 
bytes of register A. 


STA 

2000 

LDA 

2000(5:5) 

ADD 

2000(4:4) 

ADD 

2000(3:3) 

ADD 

2000(2:2) 

ADD 

2000(1:1) 


This is sometimes called “sideways addition.” 

Overflow will occur in some MIX computers when it would not occur in 
others, because of the variable definition of byte size. We have not said that 
overflow will occur definitely if the value is greater than 1073741823: overflow 
occurs when the magnitude of the result is greater than the contents of five bytes, 
depending on the byte size. One can still write programs that work properly and 
that give the same final answers, regardless of the byte size. 

• SUB (subtract). C = 2; F = field. 

V is subtracted from rA. (Equivalent to ADD but with — V in place of V.) 

• MUL (multiply). C = 3; F = field. 

The 10-byte product, V times rA, replaces registers A and X. The signs of rA 
and rX are both set to the algebraic sign of the product (namely, + if the signs 
of V and rA were the same, — if they were different). 

• DIV (divide). C = 4; F = field. 

The value of rA and rX, treated as a 10-byte number rAX with the sign of rA, 
is divided by the value V. If V = 0 or if the quotient is more than five bytes in 
magnitude (this is equivalent to the condition that |rA| > |V|), registers A and X 
are filled with undefined information and the overflow toggle is set on. Otherwise 
the quotient ±[|rAX/V|J is placed in rA and the remainder ±(|rAX| mod |V|) 
is placed in rX. The sign of rA afterwards is the algebraic sign of the quotient 
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(namely, + if the signs of V and rA were the same, — if they were different). 
The sign of rX afterwards is the previous sign of rA. 

Examples of arithmetic instructions: In most cases, arithmetic is done only 
with MIX words that are single five-byte numbers, not packed with several fields. 
It is, however, possible to operate arithmetically on packed MIX words, if some 
caution is used. The following examples should be studied carefully. (As before, 
? designates an unknown value.) 


ADD 1000 


SUB 1000 


MUL 1000 


MUL 1000(1:1) 


MUL 1000 


DIV 1000 


+ 

1234 

1 

150 

+ 

100 

5 

50 

+ 

1334 

6 

200 


- 

1234 

0 

0 

9 

- 

2000 

1 

50 

0 

+ 

766 

149 

? 


+ 

1 

1 

1 

1 

1 

+ 
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1 

1 

1 

1 

+ 

0 

1 

2 

3 

4 

+ 

5 

4 

3 

2 

1 


- 

112 
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2 ? ? ? ? 

- 

0 

- 

224 
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50 

0 

1 

12 

4 

- 

2 

0 

0 

0 

0 

+ 

1 

00 

0 

2 

24 

+ 

8 

0 

0 

0 

0 


+ 

1 1 1 1 

0 

? 

17 

+ 

3 

+ 

5 

+ 

2 


rA before 
Cell 1000 
rA after 

rA before 
Cell 1000 
rA after 

rA before 
Cell 1000 
rA after 
rX after 

rA before 
Cell 1000 
rA after 
rX after 

rA before 
Cell 1000 
rA after 
rX after 

rA before 
rX before 
Cell 1000 
rA after 
rX after 
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DIV 1000 


- 


' 0 

+ 

12 

35 

0 

3 

i 

- 

0 

0 

0 

2 

0 

+ 

0 

6 

17 

? 

? 

- 

0 

0 

0 

? 

1 


rA before 
rX before 
Cell 1000 
rA after 
rX after 


(These examples have been prepared with the philosophy that it is better to give 
a complete, baffling description than an incomplete, straightforward one.) 

Address transfer operators. In the following operations, the (possibly in- 
dexed) “address” M is used as a signed number, not as the address of a cell in 
memory. 

• ENTA (enter A). C = 48; F = 2. 

The quantity M is loaded into rA. The action is equivalent to “LDA” from a 
memory word containing the signed value of M. If M = 0, the sign of the 
instruction is loaded. 

Examples: “ENTA 0” sets r A to zeros, with a + sign. “ENTA 0 , 1” sets rA 
to the current contents of index register 1, except that —0 is changed to +0. 
“ENTA -0,1” is similar, except that +0 is changed to -0. 

• ENTX (enter X). C = 55; F = 2. 

• ENT?’ (enter i). C = 48 + i; F = 2. 

Analogous to ENTA, loading the appropriate register. 

• ENNA (enter negative A). C = 48; F = 3. 

• ENNX (enter negative X). C = 55; F = 3. 

• ENN? (enter negative i). C = 48 + i; F = 3. 

Same as ENTA, ENTX, and ENT?’, except that the opposite sign is loaded. 

Example: “ENN3 0,3” replaces rI3 by its negative, although -0 remains -0. 

• INCA (increase A). C = 48; F = 0. 

The quantity M is added to rA; the action is equivalent to “ADD” from a memory 
word containing the value of M. Overflow is possible and it is treated just as 
in ADD. 

Example: “INCA 1” increases the value of rA by one. 

• INCX (increase X). C = 55; F = 0. 

The quantity M is added to rX. If overflow occurs, the action is equivalent to 
ADD, except that rX is used instead of rA. Register A is never affected by this 
instruction. 

• INC? (increase i). C = 48 + ?’; F = 0. 

Add M to rib Overflow must not occur; if M + rl? doesn’t fit in two bytes, the 
result of this instruction is undefined. 
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• DECA (decrease A). C = 48; F = 1. 

• DECX (decrease X). C = 55; F = 1. 

• DEC* (decrease *). C = 48 + *; F = 1. 

These eight instructions are the same as INCA, INCX, and INC*, respectively, 
except that M is subtracted from the register rather than added. 

Notice that the operation code C is the same for ENTA, ENNA, INCA, and 
DECA; the F-field is used to distinguish the various operations from each other. 

Comparison operators. Mix’s comparison operators all compare the value 
contained in a register with a value contained in memory. The comparison 
indicator is then set to LESS, EQUAL, or GREATER according to whether the value 
of the register is less than, equal to, or greater than the value of the memory 
cell. A minus zero is equal to a plus zero. 

• CMPA (compare A). C = 56; F = field. 

The specified field of rA is compared with the same field of CONTENTS (M). If 
F does not include the sign position, the fields are both considered nonnegative; 
otherwise the sign is taken into account in the comparison. (An equal comparison 
always occurs when F is (0:0), since minus zero equals plus zero.) 

• CMPX (compare X). C = 63; F = field. 

This is analogous to CMPA. 

• CMP* (compare *). C = 56 + *; F = field. 

Analogous to CMPA. Bytes 1, 2, and 3 of the index register are treated as zero in 
the comparison. (Thus if F = (1:2), the result cannot be GREATER.) 

Jump operators. Instructions are ordinarily executed in sequential order; in 
other words, the command that is performed after the command in location P 
is usually the one found in location P + 1. But several “jump” instructions 
allow this sequence to be interrupted. When a typical jump takes place, the 
J-register is set to the address of the next instruction (that is, to the address of 
the instruction that would have been next if we hadn’t jumped). A “store J” 
instruction then can be used by the programmer, if desired, to set the address 
field of another command that will later be used to return to the original place 
in the program. The J-register is changed whenever a jump actually occurs in 
a program, except when the jump operator is JSJ, and it is never changed by 
non-jumps. 

• JMP (jump). C = 39; F = 0. 

Unconditional jump: The next instruction is taken from location M. 

• JSJ (jump, save J). C = 39; F = 1. 

Same as JMP except that the contents of rJ are unchanged. 

• JOV (jump on overflow). C = 39; F = 2. 

If the overflow toggle is on, it is turned off and a JMP occurs; otherwise nothing 
happens. 

• JNOV (jump on no overflow). C = 39; F = 3. 

If the overflow toggle is off, a JMP occurs; otherwise it is turned off. 
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• JL, JE, JG. JGE, JNE, JLE (jump on less, equal, greater, greater-or-equal, 
unequal, less-or-equal). C = 39; F = 4, 5, 6, 7, 8, 9, respectively. 

Jump if the comparison indicator is set to the condition indicated. For example, 
JNE will jump if the comparison indicator is LESS or GREATER. The comparison 
indicator is not changed by these instructions. 

• JAN, JAZ, JAP, JANN, JANZ, JANP (jump A negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 40; F = 0, 1, 2, 3, 4, 5, respectively. 

If the contents of rA satisfy the stated condition, a JMP occurs, otherwise nothing 
happens. "Positive” means greater than zero (not zero); “nonpositive” means 
the opposite, namely zero or negative. 

• JXN, JXZ, JXP, JXNN, JXNZ, JXNP (jump X negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 47; F = 0, 1, 2, 3, 4, 5, respectively. 

• JiN, JiZ, JiP, JiNN, Jv’NZ, JiNP (jump i negative, zero, positive, nonnegative, 
nonzero, nonpositive). C = 40 + v, F = 0, 1, 2, 3, 4, 5, respectively. These are 
analogous to the corresponding operations for rA. 


Miscellaneous operators. 

• SLA. SRA. SLAX, SRAX, SLC, SRC (shift left A, shift right A, shift left AX, shift 
right AX, shift left AX circularly, shift right AX circularly). C = 6; F = 0, 1, 2, 
3, 4, 5, respectively. 

These are the “shift” commands, in which M specifies a number of MIX bytes to 
be shifted left or right; M must be nonnegative. SLA and SRA do not affect rX; 
the other shifts affect both registers A and X as though they were a single 10- 
byte register. With SLA, SRA, SLAX, and SRAX, zeros are shifted into the register 
at one side, and bytes disappear at the other side. The instructions SLC and SRC 
call for a “circulating” shift, in which the bytes that leave one end enter in at 
the other end. Both rA and rX participate in a circulating shift. The signs of 
registers A and X are not affected in any way by any of the shift commands. 


Examples: 

Initial contents 

SRAX 1 

SLA 2 

SRC 4 

SRA 2 

SLC 501 


Register A 


Register X 


- 

6 

7 

8 

9 

10 

- 

5 

6 

7 

8 

9 

- 

5 

6 

7 

8 

9 

- 

3 

4 

0 

0 

5 

- 

3 

4 

0 

0 

5 

- 

4 

0 

0 

5 

0 


+ 

1 

2 

3 

4 

5 

+ 

0 

1 

2 

3 

4 

+ 

2 

3 

4 

0 

0 

+ 

6 

7 

8 

9 

2 

+ 

0 

0 

6 

7 

8 

+ 

0 

6 

7 

8 

3 


• MOVE. C = 7; F = number, normally 1. 

The number of words specified by F is moved, starting from location M to the 
location specified by the contents of index register 1. The transfer occurs one 
word at a time, and rll is increased by the value of F at the end of the operation. 
If F = 0, nothing happens. 

Care must be taken when there’s overlap between the locations involved; for 
example, suppose that F = 3 and M = 1000. Then if rll = 999, we transfer 
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CONTENTS (1000) to CONTENTS (999) , CONTENTS (1001) to CONTENTS (1000) . and 
CONTENTS (1002) to CONTENTS(lOOl) ; nothing unusual occurred here. But if rll 
were 1001 instead, we would move CONTENTS (1000) to CONTENTS (1001) , then 
CONTENTS (1001) to CONTENTS (1002) , then CONTENTS (1002) to CONTENTS (1003) . 
so we would have moved the same word CONTENTS (1000) into three places. 

• NOP (no operation). C = 0. 

No operation occurs, and this instruction is bypassed. F and M are ignored. 

• HLT (halt). C = 5; F = 2. 

The machine stops. When the computer operator restarts it, the net effect is 
equivalent to NOP. 

Input-output operators. MIX has a fair amount of input-output equipment 
(all of which is optional at extra cost). Each device is given a number as follows: 


Unit number 

Peripheral device 

Block size 

t 

Tape unit number t (0 < t < 7) 

100 words 

d 

Disk or drum unit number d (8 < d < 15) 

100 words 

16 

Card reader 

16 words 

17 

Card punch 

16 words 

18 

Line printer 

24 words 

19 

Typewriter terminal 

14 words 

20 

Paper tape 

14 words 


Not every MIX installation will have all of this equipment available; we will 
occasionally make appropriate assumptions about the presence of certain devices. 
Some devices may not be used both for input and for output. The number of 
words mentioned in the table above is a fixed block size associated with each unit. 

Input or output with magnetic tape, disk, or drum units reads or writes full 
words (five bytes and a sign). Input or output with units 16 through 20, however, 
is always done in a character code where each byte represents one alphameric 
character. Thus, five characters per MIX word are transmitted. The character 
code is given at the top of Table 1, which appears at the close of this section and 
on the end papers of this book. The code 00 corresponds to “ u ” , which denotes 
a blank space. Codes 01-29 are for the letters A through Z with a few Greek 
letters thrown in; codes 30-39 represent the digits 0, 1, . . . , 9; and further codes 
40, 41, ... represent punctuation marks and other special characters. (Mix's 
character set harks back to the days before computers could cope with lowercase 
letters.) We cannot use character code to read in or write out all possible values 
that a byte may have, since certain combinations are undefined. Moreover, some 
input-output devices may be unable to handle all the symbols in the character 
set; for example, the symbols E and n that appear amid the letters will perhaps 
not be acceptable to the card reader. When character-code input is being done, 
the signs of all words are set to +; on output, signs are ignored. If a typewriter is 
used for input, the “carriage return” that is typed at the end of each line causes 
the remainder of that line to be filled with blanks. 
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The disk and drum units are external memory devices each containing 100- 
word blocks. On every IN, OUT, or IOC instruction as defined below, the particular 
100-word block referred to by the instruction is specified by the current contents 
of rX, which should not exceed the capacity of the disk or drum involved. 

• IN (input). C = 36; F = unit. 

This instruction initiates the transfer of information from the input unit specified 
into consecutive locations starting with M. The number of locations transferred 
is the block size for this unit (see the table above). The machine will wait at 
this point if a preceding operation for the same unit is not yet complete. The 
transfer of information that starts with this instruction will not be complete 
until an unknown future time, depending on the speed of the input device, so a 
program must not refer to the information in memory until then. It is improper 
to attempt to read any block from magnetic tape that follows the latest block 
written on that tape. 

• OUT (output). C = 37; F = unit. 

This instruction starts the transfer of information from memory locations start- 
ing at M to the output unit specified. The machine waits until the unit is ready, 
if it is not initially ready. The transfer will not be complete until an unknown 
future time, depending on the speed of the output device, so a program must 
not alter the information in memory until then. 

• IOC (input-output control). C — 35; F = unit. 

The machine waits, if necessary, until the specified unit is not busy. Then a 
control operation is performed, depending on the particular device being used. 
The following examples are used in various parts of this book: 

Magnetic tape: If M = 0, the tape is rewound. If M < 0 the tape is skipped 
backward — M blocks, or to the beginning of the tape, whichever comes first. 
If M > 0, the tape is skipped forward; it is improper to skip forward over any 
blocks following the one last written on that tape. 

For example, the sequence “OUT 1000(3); IOC -1(3); IN 2000(3)” writes 
out one hundred words onto tape 3, then reads it back in again. Unless the 
tape reliability is questioned, the last two instructions of that sequence are only 
a slow way to move words 1000-1099 to locations 2000-2099. The sequence 
“OUT 1000(3); IOC +1(3)” is improper. 

Disk or drum: M should be zero. The effect is to position the device 
according to rX so that the next IN or OUT operation on this unit will take 
less time if it uses the same rX setting. 

Line printer: M should be zero. “IOC 0(18)” skips the printer to the top 
of the following page. 

Paper tape: M should be zero. “IOC 0(20)” rewinds the tape. 

• JRED (jump ready). C = 38; F = unit. 

A jump occurs if the specified unit is ready, that is, finished with the preceding 
operation initiated by IN, OUT, or IOC. 

• JBUS (jump busy). C = 34; F = unit. 

Analogous to JRED, but the jump occurs when the specified unit is not ready. 
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Example: In location 1000, the instruction “JBUS 1000(16)” will be exe- 
cuted repeatedly until unit 16 is ready. 


The simple operations above complete Mix’s repertoire of input-output in- 
structions. There is no “tape check” indicator, etc., to cover exceptional con- 
ditions on the peripheral devices. Any such condition (e.g., paper jam, unit 
turned off, out of tape, etc.) causes the unit to remain busy, a bell rings, and 
the skilled computer operator fixes things manually using ordinary maintenance 
procedures. Some more complicated peripheral units, which are more expensive 
and more representative of contemporary equipment than the fixed-block-size 
tapes, drums, and disks described here, are discussed in Sections 5.4.6 and 5.4.9. 

Conversion Operators. 

• NUM (convert to numeric). C = 5; F = 0. 

This operation is used to change the character code into numeric code. M is 
ignored. Registers A and X are assumed to contain a 10-byte number in character 
code; the NUM instruction sets the magnitude of rA equal to the numerical value 
of this number (treated as a decimal number). The value of rX and the sign of 
rA are unchanged. Bytes 00, 10, 20, 30, 40, ... convert to the digit zero; bytes 
01, 11, 21, . . . convert to the digit one; etc. Overflow is possible, and in this case 
the remainder modulo b 5 is retained, where b is the byte size. 

• CHAR (convert to characters). C = 5; F = 1. 

This operation is used to change numeric code into character code suitable for 
output to punched cards or tape or the line printer. The value in rA is converted 
into a 10-byte decimal number that is put into registers A and X in character 
code. The signs of rA and rX are unchanged. M is ignored. 


Examples: 


Register A 


Register X 


Initial contents 

- 

00 

00 

31 

32 

39 

NUM 0 

- 



i: 

2977 

700 

INCA 1 

- 



i: 

2977' 

699 

CHAR 0 

- 

30 

30 

31 

32 

39 


+ 

37 

57 

47 

30 

30 

+ 

37 

57 

47 

30 

30 

+ 

37 

57 

47 

30 

30 

+ 

37 

37 

36 

39 

39 


Timing. To give quantitative information about the efficiency of MIX programs, 
each of Mix’s operations is assigned an execution time typical of vintage-1970 
computers. 

ADD, SUB, all LOAD operations, all STORE operations (including STZ), all shift 
commands, and all comparison operations take two units of time. MOVE requires 
one unit plus two for each word moved. MUL, NUM, CHAR each require 10 units and 
DIV requires 12. The execution time for floating point operations is specified in 
Section 4.2.1. All remaining operations take one unit of time, plus the time the 
computer may be idle on the IN, OUT, IOC, or HLT instructions. 

Notice in particular that ENTA takes one unit of time, while LDA takes two 
units. The timing rules are easily remembered because of the fact that, except 
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for shifts, conversions. MUL, and DIV, the number of time units equals the number 
of references to memory (including the reference to the instruction itself). 

MIX’s basic unit of time is a relative measure that we will denote simply 
by u. It may be regarded as, say, 10 microseconds (for a relatively inexpensive 
computer) or as 10 nanoseconds (for a relatively high-priced machine). 

Example: The sequence LDA 1000; INCA 1; STA 1000 takes exactly 5u. 


And now I see with eye serene 
The very pulse of the machine. 
— WILLIAM WORDSWORTH, 
She Was a Phantom of Delight (1804) 


Summary. We have now discussed all the features of MIX, except for its 
"GO button,” which is discussed in exercise 26. Although MIX has nearly 150 
different operations, they fit into a few simple patterns so that they can easily 
be remembered. Table 1 summarizes the operations for each C-setting. The 
name of each operator is followed in parentheses by its default F-field. 

The following exercises give a quick review of the material in this section. 
They are mostly quite simple, and the reader should try to do nearly all of them. 


EXERCISES 

1. [00] If MIX were a ternary (base 3) computer, how many “trits” would there be 
per byte? 

2. [02] If a value to be represented within MIX may get as large as 99999999, how 
many adjacent bytes should be used to contain this quantity? 

3. [02] Give the partial field specifications, (L:R), for the (a) address field, (b) index 
field, (c) field field, and (d) operation code field of a MIX instruction. 

4. [00] The last example in (5) is “LDA -2000,4’ . How can this be legitimate, in 
view of the fact that memory addresses should not be negative? 

5. [10] What symbolic notation, analogous to (4), corresponds to (6) if (6) is re- 
garded as a MIX instruction? 

► 6. [10] Assume that location 3000 contains 


+ 

5 

1 

200 

15 


What is the result of the following instructions? (State if any of them are undefined 
or only partially defined.) (a) LDAN 3000; (b) LD2N 3000(3:4); (c) LDX 3000(1:3); 
(d) LD6 3000; (e) LDXN 3000(0:0). 

7. [Ml 5] Give a precise definition of the results of the DIV instruction for all cases 
in which overflow does not occur, using the algebraic operations X mod 1 and [X/Y\. 


8. [15] The last example of the DIV instruction that appears on page 133 has “rX 

instead, but 


before” equal to 

+ 

1235 

0 

3 

1 

. If this were 

- 

1234 

0 

3 

1 


other parts of that example were unchanged, what would registers A and X contain 
after the DIV instruction? 
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Table 1 

Character code: 


00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 

uABCDEFGHIAJKLMNOPQREnSTU 


>-4 



o 

o 

01 | 2 

02 | 2 

03 1U 

No operation 

NOP (0) 

rA 4- r A + V 

ADD (0:5) 
FADD(6) 

r A 4- rA - V 

SUB (0:5) 

FSUB (6) 

rAX ^rAxV 

MUL (0:5) 

FMUL (6) 

o 

00 

t'S 

09 | 2 

10 | 2 

11 | 2 

rA 4- V 

LDA(0 : 5) 

rll 4- V 

LD1 (0 : 5) 

rI2 4 — V 

LD2 (0 : 5) 

rI3 4- V 

LD3 (0 : 5) 

16 | 2 

17 | 2 

18 | 2 

19 2 

rA 4- -V 

LDAN (0:5) 

rll 4- -V 

LD1N(0 : 5) 

rI2 4 — -V 

LD2N (0:5) 

rI3 4- -V 

LD3N (0:5) 

24 | 2 

25 2 

26 | 2 

27 | 2 

M(F) 4- rA 

STA(0:5) 

M(F) 4 — rll 

ST1 (0:5) 

M(F) 4- rI2 

ST2 (0 : 5) 

M(F) 4- rI3 

ST3(0 : 5) 

32 | 2 

33 j 2 

34 | 1 

35 | 1 + T 

M(F) 4- rj 

STJ (0 : 2) 

M(F) 4- 0 

STZ (0 : 5) 

Unit F busy? 

JBUS(0) 

Control, unit F 

I0C(0) 

40 | 1 

41 j 1 

42 | 1 

43 | 1 

rA : 0, jump 

JA [+] 

rll : 0, jump 

JIM 

rI2 : 0, jump 

J2 [+] 

rI3 : 0, jump 

J3[+] 

48 j 1 

49 | 1 

50 | 1 

51 | 1 

rA 4- [rA]? ± M 

INCA (0) DECA(l) 
ENTA(2) ENNAC3) 

rll 4- [rll]? ± M 

INCl(O) DECl(l) 
ENT1 (2) ENN1 (3) 

rI2 4- [rI2] ? ± M 

INC2 (0) DEC2 (1) 
ENT2 (2) ENN2 (3) 

rI3 4- [rI3] ? ± M 

INC3 (0) DEC3(1) 
ENT3 (2) ENN3 (3) 

56 | 2 

57 | 2 

58 | 2 

59 | 2 

Cl 4- rA(F) : V 

CMPA (0 : 5) 
FCMPC6) 

Cl 4- rll(F) : V 

CMP1 (0 : 5) 

Cl 4- rI2(F) : V 

CMP2(0 : 5) 

Cl 44- rI3(F) : V 

CMP3(0 : 5) 


General form: 



C = operation code, (5:5) field of instruction 
F = op variant, (4:4) field of instruction 
M = address of instruction after indexing 
V = M(F) = contents of F field of location M 
OP = symbolic name for operation 
(F) = normal F setting 

t = execution time; T = interlock time 
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25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 

VWXYZ0123456789 . , ()+-*/ = $<>0; : ' 


04 f 12 

05 10 

06 

2 

07 \ 1 + 2F 

rA 4- rAX/V 

Special 

Shift M bytes 

Move F words 

rX 4 — remainder 

NUM(O) 

SLA (0) 

SRA(l) 

from M to rll 

DIV (0 : 5) 

CHAR ( 1 ) 

SLAX(2) SRAX(3) 

M0VE(1) 

FDIV (6) 

HLTC2) 

SLC (4) 

SRC (5) 

12 | 2 

13 1 2 

14 

2 

15 | 2 

rI4 4- V 

rI5 4- V 

rI6 +- V 

rX +- V 

LD4 (0 : 5) 

LD5 (0:5) 

LD6 (0 : 5) 

LDX (0 : 5) 

20 | 2 

21 | 2 

22 

2 

23 | 2 

rI4 4 V 

rI5 4- -V 

rI6 +- -V 

rX +- -V 

LD4N (0 : 5) 

LD5N (0:5) 

LD6N (0 : 5) 

LDXN (0:5) 

28 2 

29 | 2 

30 

2 

31 | 2 

M(F) 4- i l l 

M(F) +- rI5 

M(F) +- rI6 

M(F) 4- rX 

ST4 (0:5) 

ST5 (0 : 5) 

ST6 (0:5) 

STX (0:5) 

36 \ 1 + T 

37 | 1 + T 

38 

1 

39 | 1 

Input, unit F 

Output, unit F 

Unit F ready? 

Jumps 

JMP(O) JSJ(l) 

IN (0) 

OUT(O) 

JRED(O) 

J0V(2) JNOV (3) 

also [*] below 

44 1 

45 | 1 

46 

1 

47 | 1 

rI4 : 0, jump 

rI5 : 0, jump 

rI6 : 0, jump 

rX : 0, jump 

J4 [+] 

J5 [+] 

J6 [+] 

JX[+] 

52 | 1 

53 1 

54 

1 

55 | 1 

rI4 4- [rI4] ? ± M 

rI5 +- [rI5] ? ± M 

rI6 +- [rI6] ? ± M 

rX 4- [rX]?±M 

INC4 (0) DEC4(1) 

INC5 (0) DEC5(1) 

INC6(0) DEC6 ( 1 ) 

INCX(O) DECX(l) 

ENT4 (2) ENN4C3) 

ENT5 (2) ENN5 (3) 

ENT6(2) ENN6C3) 

ENTXC2) ENNX (3) 

60 | 2 

61 | 2 

62 

2 

63 | 2 

Cl 4— rI4(F) : V 

Cl 4- rI5(F) : V 

Cl +- rI6(F) : V 

Cl 4- rX(F) : V 

CMP4(0 : 5) 

CMP5 (0 : 5) 

CMP6(0:5) 

CMPX(0 : 5) 




[*] : 


[+] : 

rA 

= register A 

JL(4) 

< 

N(0) 

rX 

= register X 

JE(5) 

= 

Z(l) 

•AX 

= registers A and X as one 

JG (6) 

> 

P (2) 

rli 

= index register i, 1 < i < 6 

JGE(7) 

> 

NN(3) 

rJ 

= register .1 

JNE(8) 


NZ(4) 

Cl 

= comparison indicator 

JLE(9) 

< 

NP (5) 
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► 9. [15] List all the MIX operators that can possibly affect the setting of the overflow 
toggle. (Do not include floating point operators.) 

10. [15] List all the MIX operators that can possibly affect the setting of the compar- 
ison indicator. 

► 11. [15] List all the MIX operators that can possibly affect the setting of rll. 

12. [10] Find a single instruction that has the effect of multiplying the current con- 
tents of rI3 by two and leaving the result in rI3. 

► 13. [10] Suppose location 1000 contains the instruction “JOV 1001” . This instruction 
turns off the overflow toggle if it is on (and the next instruction executed will be in 
location 1001, in any case). If this instruction were changed to “JN0V 1001”, would 
there be any difference? What if it were changed to “JOV 1000” or “JN0V 1000”? 

14. [20] For each MIX operation, consider whether there is a way to set the ±AA, 
I, and F portions so that the result of the instruction is precisely equivalent to NOP 
(except that the execution time may be longer). Assume that nothing is known about 
the contents of any registers or any memory locations. Whenever it is possible to 
produce a NOP, state how it can be done. Examples: INCA is a no-op if the address and 
index parts are zero. JMP can never be a no-op, since it affects rJ. 

15. [10] How many alphameric characters are there in a typewriter or paper-tape 
block? in a card-reader or card-punch block? in a line-printer block? 

16. [20] Write a program that sets memory cells 0000-0099 all to zero and is (a) as 
short a program as possible; (b) as fast a program as possible. [Hint: Consider using 
the MOVE command.] 

17. [26] This is the same as the previous exercise, except that locations 0000 through 
N. inclusive, are to be set to zero, where N is the current contents of rI2. Your 
programs (a) and (b) should work for any value 0 < N < 2999; they should start in 
location 3000. 

► 18. [22] After the following “number one” program has been executed, what changes 
to registers, toggles, and memory have taken place? (For example, what is the final 
setting of rll? of rX? of the overflow and comparison indicators?) 


STZ 

1 

ENNX 

1 

STX 

1(0:1) 

SLAX 

1 

ENNA 

1 

INCX 

1 

ENT1 

1 

SRC 

1 

ADD 

1 

DEC1 

-1 

STZ 

1 

CMPA 

1 

MOVE 

-1,1(1) 

NUM 

1 

CHAR 

1 

HLT 

1 1 


► 19. [H] What is the execution time of the program in the preceding exercise, not 
counting the HLT instruction? 
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20. [20] Write a program that sets all 4000 memory cells equal to a “HLT” instruction, 
and then stops. 

► 21. [24] (a) Can the J-register ever be zero? (b) Write a program that, given a number 
N in rI4, sets register J equal to N, assuming that 0 < N < 3000. Your program should 
start in location 3000. When your program has finished its execution, the contents of 
all memory cells must be unchanged. 

► 22. [28] Location 2000 contains an integer number, X. Write two programs that 
compute X 13 and halt with the result in register A. One program should use the 
minimum number of MIX memory locations; the other should require the minimum 
execution time possible. Assume that A' 13 fits into a single word. 

23. [27] Location 0200 contains a word 



+ 

a 


c 

d 

e 

•) 

write two programs that compute the “reflected” word 


+ 

e 

d 

c 

b 

a 



and halt with the result in register A. One program should do this without using 
Mix’s ability to load and store partial fields of words. Both programs should take the 
minimum possible number of memory locations under the stated conditions (including 
all locations used for the program and for temporary storage of intermediate results) . 

24. [21] Assuming that registers A and X contain 



+ 

0 

a 

b 

c 

d 

and 

+ 

, e 

/ 

9 

h 

i 

5 

respectively, write two programs that change the contents of these registers to 


+ 

a 

T 

c 

d 

e 

and 

+ 

0 

/ 

9 

h 

i 

1 


respectively, using (a) minimum memory space and (b) minimum execution time. 

► 25. [30] Suppose that the manufacturer of MIX wishes to come out with a more 
powerful computer (“Mixmaster”?), and he wants to convince as many as possible 
of those people now owning a MIX computer to invest in the more expensive machine. 
He wants to design this new hardware to be an extension of MIX, in the sense that 
all programs correctly written for MIX will work on the new machines without change. 
Suggest desirable things that could be incorporated in this extension. (For example, 
can you make better use of the I-field of an instruction?) 

► 26. [32] This problem is to write a card-loading routine. Every computer has its own 
peculiar “bootstrapping” problems for getting information initially into the machine 
and for starting a job correctly. In Mix’s case, the contents of a card can be read only 
in character code, and the cards that contain the loading program itself must meet this 
restriction. Not all possible byte values can be read from a card, and each word read 
in from cards is positive. 

MIX has one feature that has not been explained in the text: There is a “GO 
button,” which is used to get the computer started from scratch when its memory 
contains arbitrary information. When this button is pushed by the computer operator, 
the following actions take place: 

1) A single card is read into locations 0000-0015; this is essentially equivalent to the 
instruction “IN 0(16)”. 
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2) When the card has been completely read and the card reader is no longer busy, a 
JMP to location 0000 occurs. The J-register is also set to zero. 

3) The machine now begins to execute the program it has read from the card. 

Note: MIX computers without card readers have their GO-button attached to another 
input device. But in this problem we will assume the presence of a card reader, unit 16. 
The loading routine to be written must satisfy the following conditions: 

i) The input deck should begin with the loading routine, followed by information 
cards containing the numbers to be loaded, followed by a “transfer card” that shuts 
down the loading routine and jumps to the beginning of the program. The loading 
routine should fit onto two cards. 

ii) The information cards have the following format: 

Columns 1-5, ignored by the loading routine. 

Column 6, the number of consecutive words to be loaded on this card (a num- 
ber between 1 and 7, inclusive). 

Columns 7-10, the location of word 1, which is always greater than 100 (so 
that it does not overlay the loading routine). 

Columns 11-20, word 1. 

Columns 21-30, word 2 (if column 6 > 2). 

Columns 71-80, word 7 (if column 6 = 7). 

The contents of words 1, 2, ... are punched numerically as decimal numbers. If a word 
is to be negative, a minus (“11-punch”) is overpunched over the least significant digit, 
e.g., in column 20. Assume that this causes the character code input to be 10, 11, 12, 
.... 19 rather than 30, 31, 32, . . . , 39. For example, a card that has 

ABCDE310000 123456789000000000 10000000 100 

punched in columns 1-40 should cause the following data to be loaded: 

1000: +0123456789; 1001: +0000000001; 1002: -0000000100. 

iii) The transfer card has the format TRANSOnnnn in columns 1—10, where nnnn is the 
place where execution should start. 

iv) The loading routine should work for all byte sizes without any changes to the cards 
bearing the loading routine. No card should contain any of the characters corresponding 
to bytes 20, 21, 48, 49, 50, ... (namely, the characters E, n, =,$.<,... ), since these 
characters cannot be read by all card readers. In particular, the ENT, INC, and CMP 
instructions cannot be used; they can’t necessarily be punched on a card. 

1.3.2. The MIX Assembly Language 

A symbolic language is used to make MIX programs considerably easier to read 
and to write, and to save the programmer from worrying about tedious cleri- 
cal details that often lead to unnecessary errors. This language, MIXAL (“MIX 
Assembly Language” ) , is an extension of the notation used for instructions in 
the previous section. Its main features are the optional use of alphabetic names 
to stand for numbers, and a location field to associate names with memory 
locations. 
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MIXAL can readily be comprehended if we consider first a simple example. 
The following code is part of a larger program; it is a subroutine to find the 
maximum of n elements X[l], . . . , X\n\, according to Algorithm 1.2.10M. 

Program M (Find the maximum). Register assignments: rA = m, rll = n, 
rI2 ee j. rI3 = k, X[i] = CONTENTS (X + i) . 


Assembled instructions Line no. 

01 
02 

03 

04 

05 

06 
01 
08 

09 

10 
11 
12 


3000: 

+ 

3009 

0 

2 

32 

3001: 

+ 

0 

1 

2 

51 

3002: 

+ 

3005 

0 

0 

39 

3003: 

+ 

1000 

3 

5 

56 

3004: 

+ 

3007 

0 

7 

39 

3005: 

+ 

0 

3 

2 

50 

3006 : 

+ 

1000 

3 

5 

08 

3007: 

+ 

1 

0 

1 

51 

3008: 

+ 

3003 

0 

2 

43 

3009: 

+ 

3009 

0 

0 

39 


L0C 

OP 

ADDRESS 

Times 

Remarks 

X 

EQU 

1000 




0RIG 

3000 



MAXIMUM 

STJ 

EXIT 

1 

Subroutine linkage 

INIT 

ENT3 

0,1 

1 

Ml. Initialize, k «— n. 


JMP 

CHANGEM 

1 

j n, mv- X[n], k<r-n — l. 

LOOP 

CMPA 

X , 3 

n — 1 

M3. Compare. 


JGE 

*+3 

n — 1 

To M5 if rn > X[k\. 

CHANGEM 

ENT2 

0,3 

A + l 

M4. Change m. j k. 


LDA 

X ,3 

A+l 

m <r~ X[k]. 


DEC3 

1 

n 

M5. Decrease k. 


J3P 

LOOP 

n 

M2. All tested? To M3 if A > 0. 

EXIT 

JMP 

* 

1 

Return to main program. | 


This program is an example of several things simultaneously: 

a) The columns headed “LOC”, “OP”, and “ADDRESS” are of principal interest; 
they contain a program in the MIXAL symbolic machine language, and we shall 
explain the details of this program below. 

b) The column headed “Assembled instructions” shows the actual numeric 
machine language that corresponds to the MIXAL program. MIXAL has been 
designed so that any MIXAL program can easily be translated into numeric 
machine language; the translation is usually carried out by another computer 
program called an assembly program or assembler. Thus, programmers may 
do all of their machine language programming in MIXAL, never bothering to 
determine the equivalent numeric codes by hand. Virtually all MIX programs in 
this book are written in MIXAL. 

c) The column headed “Line no.” is not an essential part of the MIXAL program; 
it is merely included with MIXAL examples in this book so that we can readily 
refer to parts of the program. 

d) The column headed “Remarks” gives explanatory information about the 
program, and it is cross-referenced to the steps of Algorithm 1.2.10M. The reader 
should compare that algorithm (page 96) with the program above. Notice that 
a little “programmer’s license” was used during the transcription into MIX code; 
for example, step M2 has been put last. The “register assignments” stated at 
the beginning of Program M show what components of MIX correspond to the 
variables in the algorithm. 

e) The column headed “Times” will be instructive in many of the MIX programs 
we will be studying in this book; it represents the profile, the number of times the 
instruction on that line will be executed during the course of the program. Thus, 
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line 06 will be performed n-1 times, etc. From this information we can determine 
the length of time required to perform the subroutine; it is (5 + 5n + 3A)u, where 
A is the quantity that was carefully analyzed in Section 1.2.10. 

Now let’s discuss the MIXAL part of Program M. Line 01, 

X EQU 1000, 

says that symbol X is to be equivalent to the number 1000. The effect of this may 
be seen on line 06, where the numeric equivalent of the instruction “CMPA X,3” 
appears as 


+ 

1000 

3 

5 

56 


that is, “CMPA 1000,3”. 

Line 02 says that the locations for succeeding lines should be chosen sequen- 
tially, originating with 3000. Therefore the symbol MAXIMUM that appears in the 
L0C field of line 03 becomes equivalent to the number 3000, INIT is equivalent 
to 3001, LOOP is equivalent to 3003, etc. 

On lines 03 through 12 the OP field contains the symbolic names of MIX 
instructions: STJ, ENT3, etc. But the symbolic names EQU and 0RIG, which 
appear in the OP column of lines 01 and 02, are somewhat different; EQU and 
0RIG are called pseudo-operations , because they are operators of MIXAL but not of 
MIX. Pseudo-operations provide special information about a symbolic program, 
without being instructions of the program itself. Thus the line 

X EQU 1000 

only talks about Program M, it does not signify that any variable is to be set 
equal to 1000 when the program is run. Notice that no instructions are assembled 
for lines 01 and 02. 

Line 03 is a “store J” instruction that stores the contents of register J into 
the (0:2) field of location EXIT. In other words, it stores rJ into the address part 
of the instruction found on line 12. 

As mentioned earlier, Program M is intended to be part of a larger program: 
elsewhere the sequence 

ENT1 100 
JMP MAXIMUM 
STA MAX 

would, for example, jump to Program M with n set to 100. Program M would 
then find the largest of the elements X[l], . . . , AF [100] and would return to the 
instruction “STA MAX” with the maximum value in rA and with its position, j , 
in rI2. (See exercise 3.) 

Line 05 jumps the control to line 08. Lines 04, 05, 06 need no further 
explanation. Line 07 introduces a new notation: An asterisk (read “self”) refers 
to the location of the line on which it appears; “*+3” (“self plus three”) therefore 
refers to three locations past the current line. Since line 07 is an instruction that 
corresponds to location 3004, the “*+3” appearing there refers to location 3007. 

The rest of the symbolic code is- self-explanatory. Notice the appearance of 
an asterisk again on line 12 (see exercise 2). 
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Our next example introduces a few more features of the assembly language. 
The object is to compute and print a table of the first 500 prime numbers, with 
10 columns of 50 numbers each. The table should appear as follows on the 
line printer: 

FIRST FIVE HUNDRED PRIMES 


0002 

0233 

0547 

0877 

1229 

1597 

1993 

2371 

2749 

3187 

0003 

0239 

0557 

0881 

1231 

1601 

1997 

2377 

2753 

3191 

0005 

0241 

0563 

0883 

1237 

1607 

1999 

2381 

2767 

3203 

0007 

0251 

0569 

0887 

1249 

1609 

2003 

2383 

2777 

3209 

0011 

0257 

0571 

0907 

1259 

1613 

2011 

2389 

2789 

3217 

0229 

0541 

0863 

1223 

1583 

1987 

2357 

2741 

3181 

3571 


We will use the following method. 

Algorithm P (Print table of 500 primes). This algorithm has two distinct 
parts: Steps P1-P8 prepare an internal table of 500 primes, and steps P9-P11 
print the answer in the form shown above. The latter part uses two “buffers,” 
in which line images are formed; while one buffer is being printed, the other one 
is being filled. 

PI. [Start table.] Set PRIME [1] t— 2, N t— 3, J <— 1. (In the following steps, 
N will run through the odd numbers that are candidates for primes; J will 
keep track of how many primes have been found so far.) 

P2. [N is prime.] Set J <- J + 1, PRIME [J] <- N. 

P3. [500 found?] If J = 500, go to step P9. 

P4. [Advance N.] Set N t— N + 2. 

P5. [K <r- 2.] Set K <— 2. (PRIME [K] will run through the possible prime divisors 
of N.) 

P6. [PRIME [K] \N?] Divide N by PRIME [K] ; let Q be the quotient and R the 
remainder. If R = 0 (hence N is not prime), go to P4. 

P7. [PRIME [K] large?] If Q < PRIME [K], go to P2. (In such a case, N must 
be prime; the proof of this fact is interesting and a little unusual — see 
exercise 6.) 

P8. [Advance K.] Increase K by 1, and go to P6. 

P9. [Print title.] Now we are ready to print the table. Advance the printer 
to the next page. Set BUFFER [0] to the title line and print this line. Set 
B <— 1, M <- 1. 

P10. [Set up line.] Put PRIME [M], PRIME [50 + M], ..., PRIME[450 + M] into 
BUFFER [B] in the proper format. 

Pll. [Print line.] Print BUFFER [B] ; set B «— 1 - B (thereby switching to the 
other buffer); and increase M by 1. If M < 50, return to P10; otherwise the 
algorithm terminates. | 
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Fig. 14. Algorithm P. 


Program P ( Print table of 500 primes). This program has deliberately been 
written in a slightly clumsy fashion in order to illustrate most of the features of 
MIXAL in a single program, rll = J — 500; rI2 = N; rI3 = K; rI4 indicates B; rI5 
is M plus multiples of 50. 

01 * EXAMPLE PROGRAM . . . TABLE OF PRIMES 


02 

* 




03 

L 

EQU 

500 

The number of primes to find 

04 

PRINTER 

EQU 

18 

Unit number of the line printer 

05 

PRIME 

EQU 

-1 

Memory area for table of primes 

06 

BUFO 

EQU 

2000 

Memory area for BUFFER [0] 

07 

BUF1 

EQU 

BUFO+25 

Memory area for BUFFER [1] 

08 


ORIG 

3000 


09 

START 

IOC 

0 (PRINTER) 

Skip to new page. 

10 


LD1 

=1-L= 

PI. Start table. J <— 1. 

11 


LD2 

=3= 

N <— 3. 

12 

2H 

INC1 

1 

P2. N is prime. J <— J + 1. 

13 


ST2 

PRIME+L , 1 

PRIME [J] N. 

U 


J1Z 

2F 

P3. 500 found? 

15 

4H 

INC2 

2 

P4. Advance N. 

16 


ENT3 

2 

P5. K <- 2. 

17 

6H 

ENTA 

0 

P6. PRIME [K]\N? 

18 


ENTX 

0,2 

rAX -f- N. 

19 


DIV 

PRIME, 3 

rA i — Q, rX t — R. 

20 


JXZ 

4B 

To P4 if R = 0. 

21 


CMPA 

PRIME, 3 

P7. PRIME [K] large? 

22 


INC3 

1 

P8. Advance K. 

23 


JG 

6B 

To P6 if Q > PRIME [K] . 

24 


JMP 

2B 

Otherwise N is prime. 
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25 

2H 

OUT 

TITLE (PRINTER) 

P9. Print title. 

26 


ENT4 

BUF1+10 

Set Bt-1. 

27 


ENT5 

-50 

Set. M <— 0. 

28 

2H 

INC5 

L+l 

Advance M. 

29 

4H 

LDA 

PRIME, 5 

P10. Set up line. (Right to left) 

30 


CHAR 


Convert PRIME [M] to decimal 

31 


STX 

0, 4(1:4) 


32 


DEC4 

1 


33 


DEC5 

50 

(rI5 goes down by 50 until 

34 


J5P 

4B 

it becomes nonpositive) 

35 


OUT 

0 ,4 (PRINTER) 

Pll. Print line. 

36 


LD4 

24,4 

Switch buffers. 

37 


J5N 

2B 

If rI5 = 0, we are done. 

38 


HLT 



39 

* INITIAL CONTENTS OF TABLES 

AND BUFFERS 

40 


0RIG 

PRIME+1 


41 


CON 

2 

The first prime is 2. 

42 


0RIG 

BUFO-5 


43 

TITLE 

ALF 

FIRST 

Alphabetic information for 

44 


ALF 

FIVE 

title line 

45 


ALF 

HUND 


46 


ALF 

RED P 


4 7 


ALF 

RIMES 


48 


0RIG 

BUFO+24 


49 


CON 

BUF1+10 

Each buffer refers to the other. 

50 


0RIG 

BUF1+24 


51 


CON 

BUF0+10 


52 


END 

START 

End of routine. | 


The following points of interest should be noted about this program: 

1. Lines 01, 02, and 39 begin with an asterisk: This signifies a “comment” 
line that is merely explanatory, having no actual effect on the assembled program. 

2. As in Program M, the pseudo-operation EQU in line 03 sets the equivalent 
of a symbol; in this case, the equivalent of L is set to 500. (In the program of lines 
10-24, L represents the number of primes to be computed.) Notice that in line 
05 the symbol PRIME gets a negative equivalent; the equivalent of a symbol may 
be any signed five-byte number. In line 07 the equivalent of BUF1 is calculated 
as BUFO+25, namely 2025. MIXAL provides a limited amount of arithmetic on 
numbers; another example appears on line 13, where the value of PRIME+L (in 
this case, 499) is calculated by the assembly program. 

3. The symbol PRINTER has been used in the F-part on lines 25 and 35. The 
F-part, which is always enclosed in parentheses, may be numeric or symbolic, 
just as the other portions of the ADDRESS field are. Line 31 illustrates the partial 
field specification “(1:4)”, using a colon. 

4. MIXAL provides several ways to specify non-instruction words. Line 41 
uses the pseudo-operation CON to specify an ordinary constant, “2”; the result 
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of line 41 is to assemble the word 



Line 49 shows a slightly more complicated constant, “BUF1+10”, which assembles 
as the word 


2035 


A constant may be enclosed in equal signs, in which case we call it a literal 
constant (see lines 10 and 11). The assembler automatically creates internal 
names and inserts “CON” lines for literal constants. For example, lines 10 and 11 
of Program P are effectively changed to 


10 LD1 coni 

11 LD2 con2 


and then at the end of the program, between lines 51 and 52, the lines 

51a coni CON 1-L 
51b con2 CON 3 


are effectively inserted as part of the assembly procedure. Line 51a will assemble 
into the word 

- T T 499 . 

The use of literal constants is a decided convenience, because it means that 
programmers do not have to invent symbolic names for trivial constants, nor 
do they have to remember to insert constants at the end of each program. 
Programmers can keep their minds on the central problems and not worry 
about such routine details. (However, the literal constants in Program P aren’t 
especially good examples, because we would have had a slightly better program 
if we had replaced lines 10 and 11 by the more efficient commands “ENT1 1-L” 
and “ENT2 3”.) 

5. A good assembly language should mimic the way a programmer thinks 
about machine programs. One example of this philosophy is the use of literal 
constants, as we have just mentioned; another example is the use of , which 
was explained in Program M. A third example is the idea of local symbols such 
as the symbol 2H, which appears in the location field of lines 12, 25, and 28. 

Local symbols are special symbols whose equivalents can be redefined as 
many times as desired. A global symbol like PRIME has but one significance 
throughout a program, and if it were to appear in the location field of more than 
one line an error would be indicated by the assembler. But local symbols have a 
different nature; we write, for example, 2H (“2 here”) in the location field, and 
2F (“2 forward”) or 2B (“2 backward”) in the address field of a MIXAL line: 

2B means the closest previous location 2H; 

2F means the closest following location 2H. 
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Thus the “2F” in line 14 refers to line 25; the “2B” in line 24 refers back to line 
12; and the “2B” in line 37 refers to line 28. An address of 2F or 2B never refers 
to its own line; for example, the three lines of MIXAL code 


2H 

EQU 

10 

2H 

MOVE 

2F(2B) 

2H 

EQU 

2B-3 


are virtually equivalent to the single line 

MOVE *-3(10). 

The symbols 2F and 2B should never be used in the location field; the symbol 
2H should never be used in the address field. There are ten local symbols, which 
can be obtained by replacing “2” in these examples by any digit from 0 to 9. 

The idea of local symbols was introduced by M. E. Conway in 1958, in 
connection with an assembly program for the UNIVAC I. Local symbols relieve 
programmers from the necessity of choosing symbolic names for every address, 
when all they want to do is refer to an instruction a few lines away. There often 
is no appropriate name for nearby locations, so programmers have tended to 
introduce meaningless symbols like XI, X2, X3, etc., with the potential danger 
of duplication. Local symbols are therefore quite useful and natural in an 
assembly language. 

6. The address part of lines 30 and 38 is blank. This means that the 
assembled address will be zero. We could have left the address blank in line 17 
as well, but the program would have been less readable without the redundant 0. 

7. Lines 43-47 use the “ALF” operation, which creates a five-byte constant 
in MIX alphameric character code. For example, line 45 causes the word 


+ 

00 

08 

24 

15 

04 


to be assembled, representing “ U HUND” — part of the title line in Program P’s 
output. 

All locations whose contents are not specified in the MIXAL program are 
ordinarily set to zero (except the locations that are used by the loading routine, 
usually 3700-3999) . Thus there is no need to set the other words of the title line 
to blanks, after line 47. 

8. Arithmetic may be used together with 0RIG: See lines 40, 42, and 48. 

9. The last line of a complete MIXAL program always has the OP-code ‘"END”. 
The address on this line is the location at which the program is to begin, once 
it has been loaded into memory. 

10. As a final note about Program P, we can observe that the instructions 
have been organized so that index registers are counted towards zero, and tested 
against zero, whenever possible. For example, the quantity J-500, not J, is kept 
in rll. Lines 26-34 are particularly noteworthy, although perhaps a bit tricky. 
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'PRINTER 


“EOT 


EOT IS 
Mr - 


* EXAMPLE PROGRAM . . . TABLE OE PR I MES 

II I I I III II I I II 
■ Ml III I I I I II I 

BBB|flBBBflBBBB66BBSBflBfl|06BBflBB60BBBB|BB6BBBBBB0BBBBBBBB66BBBB6B86BBB 

I » i « i • > • i wn VON 'H»n«ii»iinni4ij»rjii»»»j}jii4aiii,*i*4e4i4ia44*i4««7 4i4»ionuus*»iii»i«Hi»n 

ini|miiiiiii|iMiiii|Miiiiimiiiimnniiimiimiiiii 
22222111112222212222 222I|]22B22I221i|222212222222122>2222>222 

J!3!:il|l!l!!21111|||!|3!|l)l! J}]]}}]))))]])]]]]),,],, njljjj 

|4444|4B44444444|BB4B4444B44444444|B444444444444B4444444B4444 
S5|BB55S|SSSSSS55SSiSSSSi5|SiSSSSS5|SSSSSiSSSSSiSSS!S55SSSSSS 
EESESSESSCE6|SSSES6SES66fi666||SEESESB6EIISBSSB6IIE$SEES666fiSS 
iii|ii|ii?lii|mminiu?im|?immiijii!miiimimn 
IlIBBtSSlSIlSBESSSlIllBSSSSSEIISBBSBlIBSBBBtllEIEIlEtlBBSEBBE 
S1S1SS11SBS|SS|SSSSISSSSSSSS!SS1||SSSSSSS1SSSSSISISSIBSSSSS3S 



\ 

\ 



oooeoooooee 

OBOOOOOO 




U U M B M •> M « >0 )l nnNIIIITngiiN 




11111111111 

11111111 




22222222222 

22222222 




33333333333 

33333333 




44444444444 

44444444 




55555555555 

55555555 




6 S 6 6 6 6 6 6 6 6 6 

SSSIBSS6 




77777777777 

77777777 



y 

lllllllllll 

1 1 1 1 1 1 1 1 


J 

99993999999 

99993993 

J 



* EXAMPLE PROGRAM . . . TABLE OF PRIMES 

* 

L EQU 500 
PRINTER EQU 18 
PRIME EQU -1 
BUFO EQU 2000 
BUF1 Equ BUFO+25 
ORIG 3000 

START IOC 0 (PRINTER) 

LD1 =1-L= 

Fig. 15. The first lines of Program P punched onto cards, or typed on a terminal. 


It may be of interest to note a few of the statistics observed when Program P 
was actually run. The division instruction in line 19 was executed 9538 times; 
the time to perform lines 10-24 was 182144u. 


MIXAL programs can be punched onto cards or typed on a computer terminal, 
as shown in Fig. 15. The following format is used in the case of punched cards: 


Columns 1-10 
Columns 12 15 
Columns 17-80 
Columns 11, 16 


LOC (location) field; 

OP field; 

ADDRESS field and optional remarks; 
blank. 


However, if column 1 contains an asterisk, the entire card is treated as a com- 
ment. The ADDRESS field ends with the first blank column following column 16: 
any explanatory information may be punched to the right of this first blank 
column with no effect on the assembled program. (Exception: When the OP field 
is ALF, the remarks always start in column 22.) 

When the input comes from a terminal, a less restrictive format is used: 
The LOC field ends with the first blank space, while the OP and ADDRESS fields 
(if present) begin with a nonblank character and continue to the next blank: the 
special OP-code ALF is, however, followed either by two blank spaces and five 
characters of alphameric data, or by a single blank space and five alphameric 
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characters, the first of which is nonblank. The remainder of each line contains 
optional remarks. 

The MIX assembly program accepts input files prepared in this manner and 
converts them to machine language programs in loadable form. Under favorable 
circumstances the reader will have access to a MIX assembler and MIX simulator, 
on which various exercises in this book can be worked out. 

Now we have seen what can be done in MIXAL. We conclude this section by 
describing the rules more carefully, and in particular we shall observe what is not 
allowed in MIXAL. The following comparatively few rules define the language. 

1. A symbol is a string of one to ten letters and/or digits, containing at least 
one letter. Examples: PRIME, TEMP, 20BY20. The special symbols d.E, dF, and 
dB. where d is a single digit, will for the purposes of this definition be replaced 
by other unique symbols according to the “local symbol” convention described 
earlier. 


2. A number is a string of one to ten digits. Example: 00052. 

3. Each appearance of a symbol in a MIXAL program is said to be either a 
“defined symbol” or a “future reference.” A defined symbol is a symbol that has 
appeared in the LOC held of a preceding line of this MIXAL program. A future 
reference is a symbol that has not yet been defined in this way. 


4. An atomic expression is either 

a) a number, or 

b) a defined symbol (denoting the numerical equivalent of that symbol, see 
rule 13), or 

c) an asterisk (denoting the value of ©; see rules 10 and 11). 


5. An expression is either 

a) an atomic expression, or 

b) a plus or minus sign followed by an atomic expression, or 

c) an expression followed by a binary operation followed by an atomic expres- 
sion. 

The six admissible binary operations are +, *, /, //, and : . They are 

defined on numeric MIX words as follows: 


C 

C 

C 

C 

C 

C 


A+B 

LDA 

AA; 

A-B 

LDA 

AA; 

A*B 

LDA 

AA; 

A/B 

LDA 

AA; 

A//B 

LDA 

AA; 

A : B 

LDA 

AA; 

and CC 

denote lc 


ADD BB; 
SUB BB; 
MUL BB; 
SRAX 5; 
ENTX 0; 


STA CC. 
STA CC. 
STX CC. 
DIV BB; 
DIV BB; 


STA CC. 
STA CC. 


MUL =8=; SLAX 5; ADD BB; STA CC. 

, __ denote locations containing the respective values of the 

symbols A. B. and C. Operations within an expression are carried out from left 
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to right. Examples: 

-1+5 equals 4. 

-1+5*20/6 equals 4*20/6 equals 80/6 equals 13 (going from left to right). 
1/ /3 equals a MIX word whose value is approximately 6 5 /3 where 

b is the byte size; that is, a word representing the fraction | 
with an assumed radix point at the left. 

1:3 equals 11 (usually used in partial field specification). 

*-3 equals © minus three. 

*** equals © times © . 

6. An A-part (which is used to describe the address field of a MIX instruction) 
is either 

a) vacuous (denoting the value zero), or 

b) an expression, or 

c) a future reference (denoting the eventual equivalent of the symbol; see 
rule 13), or 

d) a literal constant (denoting a reference to an internally created symbol; see 
rule 12). 

7. An index part (which is used to describe the index field of a MIX instruc- 
tion) is either 

a) vacuous (denoting the value zero), or 

b) a comma followed by an expression (denoting the value of that expression). 

8. An F-part, which is used to describe the F-field of a MIX instruction, is 
either 

a) vacuous (denoting the normal F-setting, based on the OP field as shown in 
Table 1.3. 1-1), or 

b) a left parenthesis followed by an expression followed by a right parenthesis 
(denoting the value of the expression). 

9. A W-value (which is used to describe a full-word MIX constant) is either 

a) an expression followed by an F-part (in which case a vacuous F-part denotes 
(0:5)), or 

b) a W-value followed by a comma followed by a W-value of the form (a). 

A W-value denotes the value of a numeric MIX word determined as follows: 
Let the W-value have the form “Ej (Fi) ,E 2 (F 2 ) , ... ,E„(F n )”, where n > 1, 
the E’s are expressions, and the F’s are fields. The desired result is the final 
value that would appear in memory location WVAL if the following hypothetical 
program were executed: 

STZ WVAL; LDA Cj; STA WVAL(Fi); . . . ; LDA C„; STA WVAL(F n ) . 
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Here Cj, . . . , C n denote locations containing the values of expressions Ej, . . . , 
E„. Each F, must have the form 8Lj + R,; where 0 < L,; < R, < 5. Examples: 


1 

is the word 

+ 


1 

1,-1000(0:2) 

is the word 


1000 

1 

-1000(0:2) ,1 

is the word 

+ 


1 


10. The assembly process makes use of a value denoted by ® (called the 
location counter ), which is initially zero. The value of © should always be a 
nonnegative number that can fit in two bytes. When the location field of a line 
is not blank, it must contain a symbol that has not been previously defined. The 
equivalent of that symbol is then defined to be the current value of ©. 

11. After processing the LOC field as described in rule 10, the assembly 
process depends on the value of the OP field. There are six possibilities for OP: 

a) OP is a symbolic MIX operator (see Table 1 at the end of the previous section). 
The chart defines the normal C and F values for each MIX operator. In this 
case the ADDRESS should be an A-part (rule 6), followed by an index part 
(rule 7), followed by an F-part (rule 8). We thereby obtain four values: C, 
F. A, and I. The effect is to assemble the word determined by the sequence 
“LDA C; STA WORD; LDA F; STA WORD (4: 4); LDA I; STA WORD (3: 3); LDA A; 
STA W0RD(0 : 2)” into the location specified by ©, and to advance © by 1. 

b) OP is “EQU”. The ADDRESS should be a W-value (see rule 9). If the LOC field 
is nonblank, the equivalent of the symbol appearing there is set equal to the 
value specified in ADDRESS. This rule takes precedence over rule 10. The 
value of © is unchanged. (As a nontrivial example, consider the line 

BYTESIZE EQU 1(4:4) 

which allows the programmer to have a symbol whose value depends on the 
byte size. This is an acceptable situation so long as the resulting program 
is meaningful with each possible byte size.) 

c) OP is “ORIG”. The ADDRESS should be a W-value (see rule 9); the location 
counter, ©, is set to this value. (Notice that because of rule 10, a symbol 
appearing in the LOC field of an ORIG line gets as its equivalent the value of 
© before it has changed. For example, 

TABLE ORIG *+100 

sets the equivalent of TABLE to the first of 100 locations.) 

d) OP is “CON” . The ADDRESS should be a W-value; the effect is to assemble a 
word, having this value, into the location specified by ©, and to advance © 
by 1. 

e) OP is “ALF”. The effect is to assemble the word of character codes formed 
by the first five characters of the address field, otherwise behaving like CON. 
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f) OP is END . The ADDRESS should be a W-value, which specifies in its (4:5) 
field the location of the instruction at which the program begins. The END 
line signals the end of a MIXAL program. The assembler effectively inserts 
additional lines just before the END line, in arbitrary order, corresponding 
to all undefined symbols and literal constants (see rules 12 and 13). Thus a 
symbol in the LOC field of the END line will denote the first, location following 
the inserted words. 

12. Literal constants: A W-value that is less than 10 characters long may 
be enclosed between “=” signs and used as a future reference. The effect is to 
create a new symbol internally and to insert a CON line defining that symbol, 
just before the END line (see remark 4 following Program P). 

13. Every symbol has one and only one equivalent value; this is a full- 
word MIX number that is normally determined by the symbol’s appearance in 
LOC according to rule 10 or rule 11(b). If the symbol never appears in LOC. 
a new line is effectively inserted before the END line, having OP = ‘’CON” and 
ADDRESS = “0" and the name of the symbol in LOC. 

Note. The most significant consequence of the rules above is the restriction 
on future references. A symbol that has not yet been defined in the LOC field 
of a previous line may not be used except as the A-part of an instruction. In 
particular, it may not be used (a) in connection with arithmetic operations: or 
(b) in the ADDRESS field of EQU, ORIG, or CON. For example, 

LDA 2F+1 

and 

CON 3F 

are both illegal. This restriction has been imposed in order to allow more efficient 
assembly of programs, and the experience gained in writing this set of books has 
shown that it is a mild limitation that rarely makes much difference. 

Actually MIX has two symbolic languages for low-level programming: MIXAL* 
a machine-oriented language that is designed to facilitate one-pass translation 
by a very simple assembly program; and PL/MIX, which more adequately reflects 
data and control structures and which looks rather like the Remarks field of 
MIXAL programs. PL/MIX will be described in Chapter 10. 

EXERCISES — First set 

1. [00] The text remarked that “X EQU 1000” does not assemble any instruction that 

sets the value of a variable. Suppose that you are writing a MIX program in which you 
wish to set the value contained in a certain memory cell (whose symbolic name is X) 
equal to 1000. How could you write this in MIXAL? 

► 2. [10] Line 12 of Program M says “JMP where * denotes the location of that line. 
Why doesn t, th e program go into an infinite loop, endlessly repeating this instruction? 

* The author was astonished to learn in 1971 that MIXAL is also the name of a laundry 
detergent in Yugoslavia, developed for use with avtomate [automatics]. 
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► 3. [23] What is the effect of the following program, if it is used in conjunction with 
Program M? 

START IN X+1(0) 

JBUS * (0) 

ENT1 100 

1H JMP MAXIMUM 
LDX X , 1 
STA X,1 
STX X , 2 
DEC1 1 
J1P IB 
OUT X+l(l) 

HLT 

END START | 

► 4. [25] Assemble Program P by hand. (It won’t take as long as you think.) What 
are the actual numerical contents of memory, corresponding to that symbolic program? 

5. [11] Why doesn’t Program P need a JBUS instruction to determine when the line 
printer is ready? 

6. [IIM20] (a) Show that if n is not prime, n has a divisor d with 1 < d < \/n. (b) Use 
this fact to show that the test in step P7 of Algorithm P proves that N is prime. 

7. [10] (a) What is the meaning of “4B” in line 34 of Program P? (b) What effect, 
if any, would be caused if the location of line 15 were changed to “2H” and the address 
of line 20 were changed to “2B”? 

► 8. [24] What does the following program do? (Do not run it on a computer, figure 
it out by hand!) 

* MYSTERY PROGRAM 


BUF 

ORIG 

*+3000 

1H 

ENT1 

1 


ENT2 

0 


LDX 

4F 

2H 

ENT3 

0,1 

3H 

STZ 

BUF ,2 


INC2 

1 


DEC3 

1 


J3P 

3B 


STX 

BUF, 2 


INC2 

1 


INC1 

1 


CMP1 

=75= 


JL 

2B 


ENN2 

2400 


OUT 

BUF+2400,2(18) 


INC2 

24 


J2N 

*-2 


HLT 


4H 

ALF 

AAAAA 


END 

IB | 
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EXERCISES — Second set 

These exercises are short programming problems, representing typical computer 
applications and covering a wide range of techniques. Every reader is encouraged to 
choose a few of these problems, in order to get some experience using MIX as well as 
a good review of basic programming skills. If desired, these exercises may be worked 
concurrently as the rest of Chapter 1 is being read. 

The following list indicates the types of programming techniques that are involved: 

The use of switching tables for multiway decisions: exercises 9, 13, and 23. 

The use of index registers with two-dimensional arrays: exercises 10. 21, and 23. 

Unpacking characters: exercises 13 and 23. 

Integer and scaled decimal arithmetic: exercises 14, 16, and 18. 

The use of subroutines: exercises 14 and 20. 

Input buffering: exercise 13. 

Output buffering: exercises 21 and 23. 

List processing: exercise 22. 

Real-time control: exercise 20. 

Graphical display: exercise 23. 

Whenever an exercise in this book says, “write a MIX program” or “write a MIX 
subroutine,” you need only write symbolic MIXAL code for what is asked. This code 
will not be complete in itself, it will merely be a fragment of a (hypothetical) complete 
piogram. No input or output need be done in a code fragment, if the data is to be 
supplied externally; one need write only LOC, OP, and ADDRESS fields of MIXAL lines, 
together with appropriate remarks. The numeric machine language, line number, and 
“times” columns (see Program M) are not required unless specifically requested, nor 
will there be an END line. 

On the other hand, if an exercise says, “write a complete MIX program,” it implies 
that an executable program should be written in MIXAL. including in particular the 
final END line. Assemblers and MIX simulators on which such complete programs can 
be tested are widely available. 

► 9. [ 25 ] Location INST contains a MIX word that purportedly is a MIX instruction. 
Write a MIX program that jumps to location GOOD if the word has a valid C-field. 
valid ±AA-field, valid I-field, and valid F-field, according to Table 1.3. 1-1: your pro- 
gram should jump to location BAD otherwise. Remember that the test for a valid F-field 
depends on the C-field; for example, if C = 7 (MOVE), any F-field is acceptable, but if C 
= 8 (LDA), the F-field must have the form 8L + R where 0 < L < R < 5. The “±AA”- 
field is to be considered valid unless C specifies an instruction requiring a memory 
address and 1 = 0 and ±AA is not a valid memory address. 

Note: Inexperienced programmers tend to tackle a problem like this by writing a 
long series of tests on the C-field, such as “LDA C; JAZ IF; DECA 5: JAN 2F; JAZ 3F: 
DECA 2; JAN 4F; This is not good practice! The best way to make multiway 

decisions is to prepare an auxiliary table containing information that encapsulates the 
desired logic. If there were, for example, a table of 64 entries, we could write “LD1 C: 
LD1 TABLE, 1; JMP 0, 1” — thereby jumping very speedily to the desired routine. Other 
useful information can also be kept in such a table. A tabular approach to the present 
problem makes the program only a little bit longer (including the table) and greatly 
increases its speed and flexibility. 
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► 10. [31] Assume that we have a 9 x 8 matrix 


/«n 

Ul2 

Ol3 

Ol8 \ 

«21 

022 

023 

028 

V fl91 

092 

093 

O 98 / 


stored in memory so that aij is in location 1000 + 8*+ j. In memory the matrix therefore 
appears as follows: 

/ (1009) (1010) (1011) ... (1016) \ 

(1017) (1018) (1019) ... (1024) 

V (1073) (1074) (1075) ... (1080)/ 

A matrix is said to have a “saddle point” if some position is the smallest value in 
its row and the largest value in its column. In symbols, a;j is a saddle point if 

an = min a,*, = max os,,. 

l<fc<8 l<fc<9 

Write a MIX program that computes the location of a saddle point (if there is at least 
one) or zero (if there is no saddle point), and stops with this value in rll. 

11 . [M29] What is the probability that the matrix in the preceding exercise has a 
saddle point, assuming that the 72 elements are distinct and assuming that all 72! 
arrangements are equally probable? What is the corresponding probability if we assume 
instead that the elements of the matrix are zeros and ones, and that all 2 ' 2 such matrices 
are equally probable? 

12 . [HM42] Two solutions are given for exercise 10 (see page 512), and a third is 
suggested; it is not clear which of them is better. Analyze the algorithms, using each 
of the assumptions of exercise 11, and decide which is the better method. 

13 . [28] A cryptanalyst wants a frequency count of the letters in a certain code. The 
code has been punched on paper tape; the end is signaled by an asterisk. Write a 
complete MIX program that reads in the tape, counts the frequency of each character 
up to the first asterisk, and then types out the results in the form 

A 0010257 

B 0000179 

D 0794301 

etc., one character per line. The number of blanks should not be counted, nor should 
characters for which the count is zero (like C in the above) be printed. For efficiency, 
“buffer” the input: While reading a block into one area of memory you can be counting 
characters from another area. You may assume that an extra block (following the one 
that contains the terminating asterisk) is present on the input tape. 

► 14 . [31] The following algorithm, due to the Neapolitan astronomer Aloysius Lilius 
and the German Jesuit mathematician Christopher Clavius in the late 16th century, is 
used by most Western churches to determine the date of Easter Sunday for any year 
after 1582. 
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Algorithm E ( Date of Easter). Let Y be the year for which the date of Easter is 
desired. 

El. [Golden number.] Set G 4- (Y mod 19) + 1. (G is the so-called “golden 
number” of the year in the 19-year Metonic cycle.) 

E2. [Century.] Set C 4— [I'/IOOJ + 1. (When Y is not a multiple of 100, C is the 
century number; for example, 1984 is in the twentieth century.) 

E3. [Corrections.] Set X 4- L3C/4J - 12, Z 4- [(8G + 5)/25j - 5. (Here X is the 
number of years, such as 1900, in which leap year was dropped in order to 
keep in step with the sun; Z is a special correction designed to synchronize 
Easter with the moon’s orbit.) 

E4. [Find Sunday.] Set D 4— [5T/4J — — 10. (March (( — D) mod 7) will actually 

be a Sunday.) 

E5. [Epact.] Set E 4— (11G + 20 + Z — X) mod 30. If E = 25 and the golden 
number G is greater than 11, or if E = 24, then increase E by 1. (This number 
E is the epact, which specifies when a full moon occurs.) 

E6. [Find full moon.] Set N 4- 44 - E. If N < 21 then set IV 4- IV + 30. (Easter 
is supposedly the first Sunday following the first full moon that occurs on or 
after March 21. Actually perturbations in the moon’s orbit do not make this 
strictly true, but we are concerned here with the “calendar moon” rather than 
the actual moon. The IVth of March is a calendar full moon.) 

E7. [Advance to Sunday.] Set IV 4— IV + 7— ((D + N) mod 7). 

E8. [Get month.] If IV > 31, the date is (IV — 31) APRIL; otherwise the date is 

N MARCH. | 

Write a subroutine to calculate and print Easter date given the year, assuming 
that the year is less than 100000. The output should have the form ll dd MONTH, yyyyy ” 
where dd is the day and yyyyy is the year. Write a complete MIX program that uses 
this subroutine to prepare a table of the dates of Easter from 1950 through 2000. 

15. [ M30 ] A fairly common error in the coding of the previous exercise is to fail to 
realize that the quantity (11G + 20 + Z — X) in step E5 may be negative; therefore the 
positive remainder mod 30 might not be computed properly. (See CACM 5 (1962), 
556.) For example, in the year 14250 we would find G = 1, X = 95, Z = 40; so if 
we had E = — 24 instead of E = +6 we would get the ridiculous answer “42 APRIL”. 
Write a complete MIX program that finds the earliest year for which this error would 
actually cause the wrong date to be calculated for Easter. 

16. [31] We showed in Section 1.2.7 that the sum 1 + \ + § 4 becomes infinitely 

large. But if it is calculated with finite accuracy by a computer, the sum actually 
exists, in some sense, because the terms eventually get so small that they contribute 
nothing to the sum if added one by one. For example, suppose we calculate the sum 
by rounding to one decimal place; then we have 1 + 0.5 + 0.3 + 0.3 + 0.2 + 0.2 + 0.1 + 
0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 = 3 . 9 . 

More precisely, let r n ( x ) be the number x rounded to n decimal places; we define 
r n {x) = |_10"a; + |J/10 n . Then we wish to find 

S n = r„(l) + r n (|) + r n (±) -| ; 

we know that Si = 3.9, and the problem is to write a complete MIX program that 
calculates and prints S n for n = 2, 3, 4, and 5. 
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Note: There is a much faster way to do this than the simple procedure of adding 
r n (l/m), one number at a time, until r n (l/m.) becomes zero. For example, we have 
* 5 ( 1 / 771 .) = 0.00001 for all values of m from 66667 to 200000; it’s wise to avoid 
calculating 1/m all 133334 times! An algorithm along the following lines should rather 
be used: 

A. Start with mu = 1, S = 1. 

B. Set m e = nih + 1 and calculate r n (l/m e ) = r. 

C. Find mh, the largest m for which r n (\/m) = r. 

D. Add ( mn — rn e + l)r to S and return to Step B. 

17. [HM30] Using the notation of the preceding exercise, prove or disprove the formula 

limn — ►oo(*S'n+l Sn') — In 10. 

18. [25] The ascending sequence of all reduced fractions between 0 and 1 that have 
denominators < n is called the “Farey series of order n.” For example, the Farey series 
of order 7 is 

0111121231432534561 
I’ 7’ 6’ 5’ 4’ 7’ 3’ 5’ 7’ 2’ 7’ 5’ 3’ 7’ 4’ 5’ 6’ 7’ I' 

If we denote this series by xo/yo, * 1 / 2 / 1 , *2/2/2, • • . , exercise 19 proves that 


x 0 = 0, y 0 = 1; *1 = 1, yi = n; 

x k + 2 = [( 2 Ik + n)/y k +i\xk+i - x k \ 
Vk+2 = l(yk + n)/y k +i\Vk+i - Vk- 


Write a MIX subroutine that computes the Farey series of order n, by storing the values 
of Xk and y k in locations X + k, Y + k, respectively. (The total number of terms in the 
series is approximately 3n 2 /7r 2 , so you may assume that n is rather small.) 

19. [M30] (a) Show that the numbers x k and y k defined by the recurrence in the 
preceding exercise satisfy the relation Xk+iy k —XkVk+i = 1. (b) Show that the fractions 
Xk/yk are indeed the Farey series of order n, using the fact proved in (a). 

► 20. [33] Assume that Mix’s overflow toggle and X-register have been wired up to the 
traffic signals at the corner of Del Mar Boulevard and Berkeley Avenue, as follows: 


rX(2 : 2) 
rX(3 : 3) 


Del Mar traffic light 
Berkeley traffic light 


0 off, 1 green, 2 amber, 3 red; 


rX(4:4) = Del Mar pedestrian light 
rX(5:5) = Berkeley pedestrian light 


0 off, 1 “WALK” , 2 “DON’T WALK”. 


Cars or pedestrians wishing to travel on Berkeley across the boulevard must trip a 
switch that causes the overflow toggle of MIX to go on. If this condition never occurs, 
the light for Del Mar should remain green. 

Cycle times are as follows: 

Del Mar traffic light is green > 30 sec, amber 8 sec; 

Berkeley traffic light is green 20 sec, amber 5 sec. 

When a traffic light is green or amber for one direction, the other direction has a red 
light. When the traffic light is green, the corresponding WALK light is on, except that 
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22 

47 
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35 
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05 
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43 

19 
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38 
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26 

44 
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33 

02 

27 

45 

46 

15 

40 

09 

34 

03 

28 


Fig. 16. A magic square. Fig. 17. Josephus’s problem, n = 8, m = 4. 

DON’T WALK flashes for 12 sec just before a green light turns to amber, as follows: 

DON’T WALK \ secj 

_ i > repeat 8 times: 

oft | sec J 

DON’T WALK 4 sec (and remains on through amber and red cycles). 

If the overflow is tripped while the Berkeley light is green, the car or pedestrian 
will pass on that cycle, but if it is tripped during the amber or red portions, another 
cycle will be necessary after the Del Mar traffic has passed. 

Assume that one MIX time unit equals 10 p sec. Write a complete MIX program 
that controls these lights by manipulating rX, according to the input given by the 
overflow toggle. The stated times are to be followed exactly unless it is impossible to 
do so. Note: The setting of rX changes precisely at the completion of a LDX or INCX 
instruction. 

21. [28] A magic square of order n is an arrangement of the numbers 1 through n 2 in 
a square array in such a way that the sum of each row and column is n(n 2 + 1)/2, and so 
is the sum of the two main diagonals. Figure 16 shows a magic square of order 7. The 
rule for generating it is easily seen: Start with 1 just below the middle square, then go 
down and to the right diagonally — when running off the edge imagine an entire plane 
tiled with squares — until reaching a filled square; then drop down two spaces from the 
most-recently-filled square and continue. This method works whenever n is odd. 

Using memory allocated in a fashion like that of exercise 10, write a complete 
MIX program to generate the 23 x 23 magic square by the method above, and to print 
the result. [This algorithm is due to Ibn al-Haytham, who was born in Basra about 
965 and died in Cairo about 1040. Many other magic square constructions make good 
programming exercises; see W. W. Rouse Ball, Mathematical Recreations and Essays, 
revised by H. S. M. Coxeter (New York: Macmillan, 1939), Chapter 7.] 

22. [31] (The Josephus problem.) There are n men arranged in a circle. Beginning at 
a particular position, we count around the circle and brutally execute every mth man; 
the circle closes as men die. For example, the execution order when n = 8 and m = 4 is 
54613872, as shown in Fig. 17: The first man is fifth to go, the second man is fourth, 
etc. Write a complete MIX program that prints out the order of execution when n = 24, 
m = 11. Try to design a clever algorithm that works at high speed when n and m are 
large (it may save your life). Reference: W. Ahrens, Mathematische Unterhaltungen 
und Spiele 2 (Leipzig: Teubner, 1918), Chapter 15. 
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23. [ 3 7] This is an exercise designed to give some experience in the many applications 
of computers for which the output is to be displayed graphically rather than in the usual 
tabular form. In this case, the object is to “draw” a crossword puzzle diagram. 

You are given as input a matrix of zeros and 
ones. An entry of zero indicates a white square; a 
one indicates a black square. The output should be a 
diagram of the puzzle, with the appropriate squares 
numbered for words across and down. 

For example, given the matrix 

0 0 0 1 \ 

0 ' 


'l 

0 

0 

0 

Vi 



1 


2 

3 


4 


■ 

5 


6 

7 


8 


■ 



■ 

9 


1(P 


11 

12 


■ 

13 



14 






Fig. 18. Diagram corresponding 
to the matrix in exercise 23. 


the corresponding puzzle diagram would be as shown in Fig. 18. A square is numbered 
if it is a white square and either (a) the square below it is white and there is no white 
square immediately above, or (b) the square to its right is white and there is no white 
square immediately to its left. If black squares occur at the edges, they should be 
removed from the diagram. This is illustrated in Fig. 18, where the black squares at 
the corners were dropped. A simple way to accomplish this is to artificially insert rows 
and columns of — l’s at the top, bottom, and sides of the given input matrix, then to 
change every +1 that is adjacent to a —1 into a —1 until no +1 remains next to any — 1. 

The following method should be used to print the final diagram on a line printer: 
Each box of the puzzle should correspond to 5 columns and 3 rows of the output page, 
where the 15 positions are filled as follows: 

Unnumbered uuuu + Number nn nn uu + Black +++++ 

white squares: uuuu + white squares: uuuu + squares: +++++ 

+++++ +++++ +++++ 


1” squares, depending on whether there are —l’s to the right or below: 
uuuu + uuuu + uuuuu uuuuu uuuuu 

UUUU + UUUU+ uuuuu uuuuu uuuuu 

++++ UUUU+ +++ + + uuuu"*" uuuuu 


The diagram shown in Fig. 18 would then be printed as shown in Fig. 19. 


The width of a printer line — 120 characters - 
is enough to allow up to 23 columns in the cross- 
word puzzle. The data supplied as input will be a 
23 x 23 matrix of zeros and ones, each row punched 
in columns 1-23 of an input card. For example, 
the card corresponding to the matrix above would 
be punched “10000111111111111111111”. The dia- 
gram will not necessarily be symmetrical, and it 
might have long paths of black squares that are 
connected to the outside in strange ways. 


Fig. 19. Representation of Fig. 18 on a line printer. 
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1.3.3. Applications to Permutations 

In this section we shall give several more examples of MIX programs, and at the 
same time introduce some important properties of permutations. These inves- 
tigations will also bring out some interesting aspects of computer programming 
in general. 

Permutations were discussed earlier in Section 1.2.5; we treated the per- 
mutation cdfbea as an arrangement of the six objects a, b , c, d, e, f in a 
straight line. Another viewpoint is also possible: We may think of a permutation 
as a rearrangement or renaming of the objects. With this interpretation it is 
customary to use a two-line notation, for example, 

fab c d e f\ 

\c d f b e a J ’ 

to mean “a becomes c, b becomes d , c becomes /, d becomes b , e becomes e, 
/ becomes a.” Considered as a rearrangement, this means that object c moves 
to the place formerly occupied by object a; considered as a renaming, it means 
that object a is renamed c. The two-line notation is unaffected by changes in 
the order of the columns; for example, the permutation (i) could also be written 

f c d f b a e\ 

\f b a d c e J 

and in 718 other ways. 

A cycle notation is often used in connection with this interpretation. Permu- 
tation (i) could be written 

( acf){bd ), ( 2 ) 

again meaning “a becomes c, c becomes /. / becomes a, b becomes d, d be- 
comes 6.” A cycle (xy x 2 ■ ■ . x n ) means “xy becomes x 2 , . . . , x n -y becomes x n , 
x n becomes Xy” Since e is fixed under the permutation, it does not appear 
in the cycle notation; that is, singleton cycles like “(e)” are conventionally not 
written. If a permutation fixes all elements, so that there are only singleton 
cycles present, it is called the identity permutation, and we denote it by “()”. 

The cycle notation is not unique. For example, 

(' bd){acf ), (cf a) (bd), ( db){fac ), (3) 

etc., are all equivalent to (2). However, “( afc)(bd )” is not the same, since it 
says that a goes to /. 

It is easy to see why the cycle notation is always possible. Starting with 
any element xy, the permutation takes xy into x 2 , say, and x 2 into x 3 , etc., until 
finally (since there are only finitely many elements) we get to some element x n+ y 
that has already appeared among xy,...,x n . Now x n+1 must equal xy. For if it 
were equal to, say, x 3 , we already know that x 2 goes into x 3 ; but by assumption, 
x n 7 ^ x 2 goes to X n+ y. So x n+1 = xy, and we have a cycle (xy x 2 ... x n ) as 
part of our permutation, for some n > 1. If this does not account for the entire 
permutation, we can find another element yy and get another cycle (yy y 2 . . . y m ) 
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in the same way. None of the y's can equal any of the x’s, since Xi — yj implies 
that Xi+i = Vj+i, etc., and we would ultimately find Xk — y 1 for some k, 
contradicting the choice of y\. All cycles will eventually be found. 

One application of these concepts to programming comes up whenever some 
set of n objects is to be put into a different order. If we want to rearrange 
the objects without moving them elsewhere, we must essentially follow the cycle 
structure. For example, to do the rearrangement (1), namely to set 

(a, b, c, d, e, /) 4- (c, d, /, b, e, a), 

we would essentially follow the cycle structure (2) and successively set 


t 4 — (l . a 4 — C, C 4 — /, f 4 — tj t i — fe, b 4 — d, d 4 t. 

It is frequently useful to realize that any such transformation takes place in 
disjoint cycles. 


Products of permutations. We can multiply two permutations together, with 
the understanding that multiplication means the application of one permutation 
after the other. For example, if permutation (1) is followed by the permutation 


a b c d e f 
b d c a f e 

we have a becomes c, which then becomes c; b becomes d, which becomes a; etc.: 


a b c d e f 
c d f b e a 


a b c d e f 
b d c a f e 

a b c d e f 
c d f b e a 

a b c d e f 
c a e d f b 


c d f b e a 
c a e d f b 


(4) 


It should be clear that multiplication of permutations is not commutative; 
in other words, 7Ti x 7 T 2 is not necessarily equal to 7T2 x tx\ when 7Ti and 7T2 are 
permutations. The reader may verify that the product in (4) gives a different 
result if the two factors are interchanged (see exercise 3). 

Some people multiply permutations from right to left rather than the some- 
what more natural left-to-right order shown in (4). In fact, mathematicians are 
divided into two camps in this regard; should the result of applying transforma- 
tion Tj, then T2, be denoted by T1T2 or by T2T1? Here we use T1T2. 

Equation (4) would be written as follows, using the cycle notation: 

(ac f) (bd) (abd) (e f) = (ace f b). (5) 

Note that the multiplication sign “x” is conventionally dropped; this does not 
conflict with the cycle notation since it is easy to see that the permutation 
( acf)(bd ) is really the product of the permutations ( acf ) and (fed). 
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Multiplication of permutations can be done directly in terms of the cycle 
notation. For example, to compute the product of several permutations 

( acfg ) (bed) ( aed ) (fade) (bgfae), (6) 

we find (proceeding from left to right) that “a goes to c, then c goes to d, then 
d goes to a, then a goes to d, then d is unchanged”; so the net result is that 
a goes to d under (6), and we write down “(ad” as the partial answer. Now 
we consider the effect on d: “d goes to b goes to g" ; we have the partial result 
“(ad g” . Considering g, we find that “ g goes to a, to e, to /, to a”, and so the 
first cycle is closed: “( adg )”. Now we pick a new element that hasn’t appeared 
yet, say c; we find that c goes to e, and the reader may verify that ultimately 
the answer “(a d g)(ceb)” is obtained for (6). 

Let us now try to do this process by computer. The following algorithm 
formalizes the method described in the preceding paragraph, in a way that is 
amenable to machine calculation. 



Fig. 20. Algorithm A for multiplying permutations. 


Algorithm A (Multiply permutations in cycle form). This algorithm takes a 
product of cycles, such as (6), and computes the resulting permutation in the 
form of a product of disjoint cycles. For simplicity, the removal of singleton cycles 
is not described here; that would be a fairly simple extension of the algorithm. 
As this algorithm is performed, we successively “tag” the elements of the input 
formula; that is, we mark somehow those symbols of the input formula that have 
been processed. 

Al. [First pass.] Tag all left parentheses, and replace each right parenthesis by 
a tagged copy of the input symbol that follows its matching left parenthesis. 
(See the example in Table 1.) 

A2. [Open.] Searching from left to right, find the first untagged element of the 
input. (If all elements are tagged, the algorithm terminates.) Set START 
equal to it; output a left parenthesis; output the element; and tag it. 

A3. [Set CURRENT.] Set CURRENT equal to the next element of the formula. 

A4. [Scan formula.] Proceed to the right until either reaching the end of the 
formula, or finding an element equal to CURRENT; in the latter case, tag it 
and go back to step A3. 
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Table 1 

ALGORITHM A APPLIED TO (6) 


After 


step 

START 

CURRENT 

(ac f g) 

(bed) 

( a e d ) 

(fade 

) ( b g fae 

) Output 

A1 



a c f g 

bed 

a e d 

fade 

b g f a e 


A2 

a 


> S H 

bed 

a e d 

fade 

b g f ae 

(a 

A3 

a 

c 

' C I 'fo 

bed 

a e d 

fade 

b g f a e 


A4 

a 

c 

c / g 

b^d 

a e d 

fade 

b g f a e 


A4 

a 

d 

cfg 

b d 

' ae 'I 

fade 

b g f ae 


A4 

a 

a 

cfg 

b d 

a e 

f l de 

b g f a e 


A5 

a 

d 

cfg 

b d 

a e 

f d e 

b g f a e 

$ d 

A 5 

a 

9 

cfg 

b 

a e 

f de 

g f a e 

T * 

A5 

a 

a 

cf 

b 

?■ ( a e d (. 

f d 

g ae 

# 

A6 

a 

a 

cf 

b 

) ( a e d < 

f d 

g ae 

C ) 

A2 

c 

a 

1/ -■ 

b 

) i o- e d ( 

f d 

g ae 

(c 

A5 

c 

e 

/ 

b 

) { a e d ( 

f a d t. 

g :■ e 

■T 

u e 

A5 

c 

b 

/ 

b 



g 

C. * 

A6 

c 

c 

/ 




g 

T ) 

A6 

f 

f 






€ (/) 


Here J represents a cursor following the element just scanned; tagged elements are light gray. 


A5. [CURRENT = START?] If CURRENT / START, output CURRENT and go back to 
step A4 starting again at the left of the formula (thereby continuing the 
development of a cycle in the output). 

A6. [Close.] (A complete cycle in the output has been found.) Output a right 
parenthesis, and go back to step A2. | 

For example, consider formula (6); Table 1 shows successive stages in its 
processing. The first line of that table shows the formula after right parentheses 
have been replaced by the leading element of the corresponding cycle; succeeding 
lines show the progress that is made as more and more elements are tagged. 
A cursor shows the current point of interest in the formula. The output is 
u (adg)(ceb)(f)”; notice that singleton cycles will appear in the output. 

A MIX program. To implement this algorithm for MIX, the “tagging” can be 
done by using the sign of a word. Suppose our input is punched onto cards in the 
following format: An 80-column card is divided into 16 five-character fields. Each 
field is either (a) “ U uuu(”, representing the left parenthesis beginning a cycle; 
(b) “Iuliuu”, representing the right parenthesis ending a cycle; (c) “uuuuu”, 
all blanks, which may be inserted anywhere to fill space; or (d) anything else, 
representing an element to be permuted. The last card of the input is recognized 
by having columns 76 80 equal to “ U uuu=”- For example, (6) might be punched 
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on two cards as follows: 


( A 

C 

F 

G ) 

( B 

C 

D 

) 

( A 

E 

D ) 


( F 

A 

D 

E ) 

( B 

G 

F 

A 

E ) 



= 


The output of our program will consist of a verbatim copy of the input, followed 
by the answer in essentially the same format. 

Program A ( Multiply permutations in cycle form). This program implements 


Maximum length of input 
The input permutation 
Place for answer 
Place for printing 
Unit number for card reader 
Unit number for printer 
Read first card. 


Wait for cycle complete. 

Is it the last card? 

No, read another. 

Print a copy of 
the input card. 


Repeat until input is complete. 

Too much input! 

At this point, rI2 words of 

input are in PERM, PERM + 1, . . . 
Al. First pass. 

Get next element of input. 

Is it “(”? 

If so, tag it. 

Put the next nonblank input symbol 
into rX. 


Replace “)” by tagged rX. 

Have all elements been processed? 


of singleton 

cycles. 

But it doesn’t cat.cl 

01 

MAXWDS 

EQU 

1200 


02 

PERM 

ORIG 

*+MAXWDS 


03 

ANS 

ORIG 

♦+MAXWDS 


04 

OUTBUF 

ORIG 

*+24 


05 

CARDS 

EQU 

16 


06 

PRINTER EQU 

18 


07 

BEGIN 

IN 

PERM (CARDS) 


08 


ENT2 

0 


09 


LDA 

EQUALS 


10 

1H 

JBUS 

* (CARDS) 


11 


CMP A 

PERM+15 , 2 


12 


JE 

*+2 


13 


IN 

PERM+16 , 2 (CARDS) 

14 


ENT1 

OUTBUF 


15 


JBUS 

* (PRINTER) 


16 


MOVE 

PERM, 2 (16) 


17 


OUT 

OUTBUF (PRINTER) 

18 


JE 

IF 


19 


INC2 

16 


20 


CMP 2 

=MAXWDS-16= 


21 


JLE 

IB 


22 


HLT 

666 


23 

1H 

INC2 

15 

1 

24 


ST2 

SIZE 

1 

25 


ENT3 

0 

1 

26 

2H 

LDAN 

PERM, 3 

A 

27 


CMPA 

LPREN(1 : 5) 

A 

28 


JNE 

IF 

A 

29 


STA 

PERM , 3 

B 

30 


INC3 

1 

B 

31 


LDXN 

PERM, 3 

B 

32 


JXZ 

*-2 

B 

33 

1H 

CMPA 

RPREN(1 : 5) 

C 

34 


JNE 

*+2 

C 

35 


STX 

PERM, 3 

D 

36 


INC3 

1 

C 

37 


CMP3 

SIZE 

C 

38 


JL 

2B 

C 
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39 


LDA 

LPREN 

1 

40 


ENT1 

ANS 

1 

41 

OPEN 

ENT3 

0 

E 

42 

1H 

LDXN 

PERM, 3 

F 

43 


JXN 

GO 

F 

44 


INC3 

1 

G 

45 


CMP3 

SIZE 

G 

46 


JL 

IB 

G 

47 

* 




48 

DONE 

CMP1 

=ANS= 


49 


JNE 

*+2 


50 


MOVE 

LPREN (2) 


51 


MOVE 

=0= 


52 


MOVE 

-1,1(22) 


53 


ENT3 

0 


54 


OUT 

ANS, 3 (PRINTER) 


55 


INC3 

24 


56 


LDX 

ANS, 3 


57 


JXNZ 

*-3 


58 


HLT 



59 

* 




60 

LPREN 

ALF 

( 


61 

RPREN 

ALF 

) 


62 

EQUALS 

ALF 

= 


63 

* 




64 

GO 

MOVE 

LPREN 

H 

65 


MOVE 

PERM , 3 

H 

66 


STX 

START 

H 

67 

SUCC 

STX 

PERM, 3 

J 

68 


INC3 

1 

J 

69 


LDXN 

PERM, 3(1 :5) 

J 

70 


JXN 

IF 

J 

71 


JMP 

*-3 

0 

72 

5H 

STX 

0,1 

Q 

73 


INC1 

1 

Q 

74 


ENT3 

0 

Q 

75 

4H 

CMPX 

PERM, 3(1:5) 

K 

76 


JE 

SUCC 

K 

77 

1H 

INC3 

1 

L 

78 


CMP3 

SIZE 

L 

79 


JL 

4B 

L 

80 


CMPX 

START (1 : 5) 

P 

81 


JNE 

5B 

P 

82 

CLOSE 

MOVE 

RPREN 

R 

83 


CMP A 

-3,1 

R 

84 


JNE 

OPEN 

R 

85 


INC1 

-3 

S 

86 


JMP 

OPEN 

S 

87 


END 

BEGIN 



Prepare for main program, 
rll = place to store next answer 
A 2. Open. 

Look for untagged element. 


All are tagged. Now comes the output. 

Is answer the identity permutation? 

If so, change to “()”. 

Put 23 words of blanks after answer. 


Print as many lines as necessary. 


Constants used in the program 


Open a cycle in the output. 


Tag an element. 

Move one step to the right. 

A3. Set CURRENT (namely rX). 
Skip past blanks. 

Output CURRENT. 

Scan formula again. 

A4. Scan formula. 

Element = CURRENT? 

Move to right. 

End of formula? 

A5. CURRENT - START? 

A6. Close. 

Note: rA = “(”. 

Suppress singleton cycles. 

I 
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This program of approximately 75 instructions is quite a bit longer than 
the programs of the previous section, and indeed it is longer than most of the 
programs we will meet in this book. Its length is not formidable, however, since 
it divides into several small parts that are fairly independent. Lines 07 22 read 
in the input cards and print a copy of each card; lines 23-38 accomplish step 
A1 of the algorithm, the preconditioning of the input; lines 39-46 and 64-86 
do the main business of Algorithm A; and lines 48-57 output the answer. The 
reader will find it instructive to study as many of the MIX programs given in 
this book as possible — it is exceedingly important to acquire skill in reading 
other people’s computer programs, yet such training has been sadly neglected 
in too many computer courses and some horribly inefficient uses of computing 
machinery have arisen as a result. 

Timing. The parts of Program A that are not concerned with input-output 
have been decorated with frequency counts, as we did for Program 1.3.2M; thus, 
line 30 is supposedly executed B times. For convenience it has been assumed 
that no blank words appear in the input except at the extreme right end; under 
this assumption, line 71 is never executed and the jump in line 32 never occurs. 

By simple addition the total time to execute the program is 

{7+5A + 6B + 7C + 2D + E + 3F + 4G + 8H + 6J 

+ 3K + 4L + 3P + 4Q + 6R + 2S)u, ( 7 ) 

plus the time for input and output. In order to understand the meaning of 
formula ( 7 ), we need to examine the fifteen unknowns A, B, C, D, E, F, G, H, 
J, K, L , P, Q , R. S and we must relate them to pertinent characteristics of the 
input. We will now illustrate the general principles of attack for problems of this 
kind. 

First we apply “Kirchhoff’s first law” of electrical circuit theory: The number 
of times an instruction is executed must equal the number of times we transfer 
to that instruction. This seemingly obvious rule often relates several quantities 
in a nonobvious way. Analyzing the flow of Program A, we get the following 
equations. 


From lines 

We deduce 

26, 38 

A = 1 + (C- 1) 

33, 28 

C — B + (A - B) 

41, 84, 86 

E =1 + R 

42, 46 

F = E+{G- 1) 

64, 43 

H — F - G 

67, 70, 76 

J = H + (K - (L- J)) 

75, 79 

K = Q + (L-P ) 

82, 72 

R = P-Q 


The equations given by Kirchhoff’s law will not all be independent; in the present 
case, for example, we see that the first and second equations are obviously 
equivalent. Furthermore, the last equation can be deduced from the others, 
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since the third, fourth, and fifth imply that H = R; hence the sixth says that 
K = L — R. At any rate we have already eliminated six of our fifteen unknowns: 

A = C, E = R+ 1, F = R + G , H = R, K = L - R, Q = P - R. (8) 

Kirchhoff’s first law is an effective tool that is analyzed more closely in Section 
2.3.4. 1. 

The next step is to try to match up the variables with important character- 


istics of the data. We find from lines 24, 25, 30, and 36 that 

B + C = number of words of input = 16X — 1, (g) 

where X is the number of input cards. From line 28, 

B = number of “(” in input = number of cycles in input. (10) 

Similarly, from line 34, 

D — number of “)” in input = number of cycles in input. (n) 

Now (10) and (n) give us a fact that could not be deduced by Kirchhoff’s law: 

B = D. (12) 

From line 64, 

H — number of cycles in output (including singletons). (13) 


Line 82 says R is equal to this same quantity; the fact that H = R was in this 
case deducible from Kirchhoff’s law, since it already appears in (8). 

Using the fact that each nonblank word is ultimately tagged, and lines 29, 
35, and 67, we find that 

J = Y-2B, (14) 

where Y is the number of nonblank words appearing in the input permutations. 
From the fact that every distinct element appearing in the input permutation is 
written into the output just once, either at line 65 or line 72, we have 

P = H + Q — number of distinct elements in input. (15) 

(See Eqs. (8).) A moment’s reflection makes this clear from line 80 as well. 
Finally, we see from line 85 that 

S = number of singleton cycles in output. (16) 

Clearly the quantities 73, C, H, J, P, and S that we have now interpreted 
are essentially independent parameters that may be expected to enter into the 
timing of Program A. 

The results we have obtained so far leave us with only the unknowns G 
and L to be analyzed. For these we must use a little more ingenuity. The scans 
of the input that start at lines 41 and 74 always terminate either at line 47 (the 
last time) or at line 80. During each one of these P + 1 loops, the instruction 
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“INC3 1” is performed B + C times; this takes place only at lines 44, 68, and 77, 
so we get the nontrivial relation 

G + J + L = (5 + C) ( P + 1) (17) 

connecting our unknowns G and L. Fortunately, the running time (7) is a func- 
tion of G + L (it involves f3F + 4G+' • -+3K + 4L + - ■ ■ — f 7G + 7L + - • ■ ), 

so we need not try to analyze the individual quantities G and L any further. 

Summing up all these results, we find that the total time exclusive of input- 
output comes to 

(112 NX + 304X -2M-Y + 11U + 2V - ll)u; (18) 

in this formula, new names for the data characteristics have been used as follows: 

X = number of cards of input, 

Y = number of nonblank fields in input (excluding filial “=” ) , 

M — number of cycles in input, 

N — number of distinct element names in input, 

U = number of cycles in output (including singletons), 

V = number of singleton cycles in output. 

In this way we have found that analysis of a program like Program A is in many 
respects like solving an amusing puzzle. 

We will show below that, if the output permutation is assumed to be random, 
the quantities U and V will be Hjv and 1, respectively, on the average. 

Another approach. Algorithm A multiplies permutations together much as 
people ordinarily do the same job. Quite often we find that problems to be solved 
by computer are very similar to problems that have confronted humans for many 
years; therefore time-honored methods of solution, which have evolved for use 
by mortals such as we, are also appropriate procedures for computer algorithms. 

Just as often, however, we encounter new methods that turn out to be 
superior for computers, although they are quite unsuitable for human use. The 
central reason is that a computer “thinks” differently; it has a different kind of 
memory for facts. An instance of this difference may be seen in our permutation- 
multiplication problem: Using the algorithm below, a computer can do the 
multiplication in one sweep over the formula, remembering the entire current 
state of the permutation as its cycles are being multiplied. The human-oriented 
Algorithm A scans the formula many times, once for each element of the output, 
but the new algorithm handles everything in one scan. This is a feat that could 
not be done reliably by Homo sapiens. 

What is this computer-oriented method for permutation multiplication? 
Table 2 illustrates the basic idea. The column below each character of the cycle 
form in that table says what permutation is represented by the partial cycles to 
the right ; for example, the fragmentary formula . d e)(b g f a e)” represents 
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Table 2 

MULTIPLYING PERMUTATIONS IN ONE PASS 
( a c f 9 ) {bed) ( a e d ) {fade) ( b g f a e ) 
a — > d d a a a a aaaaaaaddddddeeeeeeeea a 
b -A ccccccccggggggggggggggggbbbbb 
c — > eeeddddddcccccccccccccccccccc 
d — » ggggggg) ) ) d d ) ) ) bbbbbddddddddd 
e -A bbbbbbbbbbbbbbaaa) ) ) ) b b ) ) ) ) ) e 
f —> ffffeeeeeeeeeeeeeeaaaaaaaafff 
9 ■“)))’ f f f f f f f f f f f f f f } f f f f f g g g g 


the permutation 

( a b c d e f q\ 

\e g c b 7 a /)’ 

which appears under the rightmost d of the table. 

Inspection of Table 2 shows that it can be created systematically, if we 
start with the identity permutation on the right and work backward from right 
to left. The column below letter x differs from the column to its right (which 
records the previous status) only in row x; and the new value in row x is the one 
that disappeared in the preceding change. More precisely, we have the following 
algorithm: 

Algorithm B (Multiply permutations in cycle form). This algorithm accom- 
plishes essentially the same result as Algorithm A. Assume that the elements per- 
muted are named xi,X 2 , ■ ■ . ,x n . We use an auxiliary table T[1],T[2], . . . ,T[nj; 
upon termination of this algorithm, Xi goes to Xj under the input permutation 
if and only if T[i\ = j. 

Bl. [Initialize.] Set T[k\ <— k for 1 < k < n. Also, prepare to scan the input 
from right to left. 

B2. [Next element.] Examine the next element of the input (right to left). If 
the input has been exhausted, the algorithm terminates. If the element is a 
“)”, set Z 0 and repeat step B2; if it is a “(”, go to B4. Otherwise the 
element is Xj for some i; go on to B3. 

B3. [Change T[i].] Exchange Z •<->• T[i], If this makes T[i] = 0, set j i. Return 
to step B2. 

B4. [Change T[j].] Set T[j] Z. (At this point, j is the row that shows a “)” 
entry in the notation of Table 2, corresponding to the right parenthesis that 
matches the left parenthesis just scanned.) Return to step B2. | 

Of course, after this algorithm has been performed, we still must output the 
contents of table T in cycle form; this is easily done by a “tagging” method, as 
we shall see below. 
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Fig. 21. Algorithm B for multiplying permutations. 


Let us now write a MIX program based on the new algorithm. We wish to use 
the same ground rules as those in Program A, with input and output in the same 
format as before. A slight problem presents itself; namely, how can we implement 
Algorithm B without knowing in advance what the elements x±, X 2 , ■ ■ . fX n are? 
We don’t know n, and we don’t know whether the element named b is to be 
X \ , or X 2 , etc. A simple way to solve this problem is to maintain a table of the 
element names that have been encountered so far, and to search for the current 
name each time (see lines 35-44 in the program below). 

Program B ( Same effect as Program A). rX = Z ; rI4 = i; rll = j; rI3 = n, the 
number of distinct names seen. 


01 

MAXWDS 

EQU 

1200 


Maximum length of input 

02 

X 

ORIG 

++MAXWDS 

The table of names 

03 

T 

ORIG 

♦+MAXWDS 

The auxiliary state table 

04 

PERM 

ORIG 

♦+MAXWDS 

The input permutation 

05 

ANS 

EQU 

PERM 


Place for answer 

06 

OUTBUF 

ORIG 

*+24 


Place for printing 

07 

CARDS 

EQU 

16 1 

> Same 

as lines 05-22 of Program A 

24 


HLT 

666 J 

1 

At this point, rI2 words of 

25 

1H 

INC2 

15 

1 

input are in PERM, PERM +1, ... 

26 


ENT3 

0 

1 

and we haven’t seen any names yet. 

27 

RIGHT 

ENTX 

0 

A 

Set Z <— 0. 

28 

SCAN 

DEC2 

1 

B 

B2. Next element. 

29 


LDA 

PERM, 2 

B 


30 


JAZ 

CYCLE 

B 

Skip over blanks. 

31 


CMPA 

RPREN 

C 


32 


JE 

RIGHT 

C 

Is the next element “)”? 

33 


CMPA 

LPREN 

D 


34 


JE 

LEFT 

D 

Is it “(”? 

35 


ENT4 

1,3 

E 

Prepare for the search. 

36 


STA 

X 

E 

Store at beginning of table. 

37 

2H 

DEC4 

1 

F 

Search through names table. 

38 


CMPA 

X , 4 

F 


39 


JNE 

2B 

F 

Repeat until match found. 

40 


J4P 

FOUND 

G 

Has the name appeared before? 

41 


INC3 

1 

H 

No; increase the table size. 

42 


STA 

X,3 

H 

Insert the new name x n . 
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43 


ST3 

T,3 

H 

Set T[n ] «— n, 

44 


ENT4 

0,3 

H 

i <— n. 

45 

FOUND 

LDA 

T , 4 

J 

B3. Change T\i], 

46 


STX 

T , 4 

J 

Store Z. 

47 


SRC 

5 

J 

Set Z. 

48 


JANZ 

SCAN 

J 


49 


ENT1 

0,4 

K 

If Z was zero, set j i. 

50 


JMP 

SCAN 

K 


51 

LEFT 

STX 

T , 1 

L 

B4. Change T\i], 

52 

CYCLE 

J2P 

SCAN 

P 

Return to B2, unless finished. 

53 

* 





54 

OUTPUT 

ENT1 

ANS 

1 

All input has been scanned. 

55 


J3Z 

DONE 

1 

The x and T tables contain the answer 

56 

1H 

LDAN 

X , 3 

Q 

Now we construct cycle notation. 

57 


JAP 

SKIP 

Q 

Has name been tagged? 

58 


CMP3 

T , 3 

R 

Is there a singleton cycle? 

59 


JE 

SKIP 

R 


60 


MOVE 

LPREN 

S 

Open a cycle. 

61 

2H 

MOVE 

X , 3 

T 


62 


STA 

X,3 

T 

Tag the name. 

63 


LD3 

T , 3 

T 

Find successor of element. 

64 


LDAN 

X , 3 

T 


65 


JAN 

2B 

T 

Is it already tagged? 

66 


MOVE 

RPREN 

W 

Yes, cycle closes. 

67 

SKIP 

DEC3 

1 

z 

Move to next name. 

68 


J3P 

IB 

z 


69 

* 





70 

DONE 

CMP1 

=ANS= ] 

\ 





1 

> Same as lines 48-62 of Program A 

84 

EQUALS 

ALF 

■ J 

i 


85 


END 

BEGIN 

i 



Lines 54-68, which construct the cycle notation from the T table and the table 
of names, make a rather pretty little algorithm that merits some study. The 
quantities A, B, . . . , R, S, T, W, Z that enter into the timing of this program 
are, of course, different, from the quantities of the same name in the analysis of 
Program A. The reader will find it an interesting exercise to analyze these times 
(see exercise 10). 

Experience shows that the main portion of the execution time of Program B 
will be spent in searching the names table this is quantity F in the timing. 
Much better algorithms for searching and building dictionaries of names are 
available; they are called symbol table algorithms, and they are of great impor- 
tance in computer applications. Chapter 6 contains a thorough discussion of 
efficient symbol table algorithms. 

Inverses. The inverse n~ of a permutation it is the rearrangement that undoes 
the effect of 7 r; if i goes to j under 7r, then j goes to i under 7r~. Thus the 
product 7T7r — equals the identity permutation, and so does the product 7r - 7r. 
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People often denote the inverse by 7r _1 instead of , but the superscript 1 is 
redundant (for the same reason that x 1 = x). 

Every permutation has an inverse. For example, the inverse of 

( a b c d e f\ . / c d f b e a\ _ ( a b c d e f\ 

\c d f b e a ) 1S \a b c d e f ) ~ \f d a b e c ) ' 

We will now consider some simple algorithms for computing the inverse of a 
permutation. 

In the rest of this section, let us assume that we are dealing with permuta- 
tions of the numbers {1,2,..., n}. If AT[1] X[2\ . . . X[n } is such a permutation, 
there is a simple method to compute its inverse: Set E[X[A]] <— k for 1 < k < n. 
Then Y[ 1] Y[2] . . . Y[n ] is the desired inverse. This method uses 2 n memory cells, 
namely n for X and n for Y. 

Just for fun, however, let’s suppose that n is very large and suppose also 
that we wish to compute the inverse of AT[1] X [2] . . . X[n] without using much 
additional memory space. We want to compute the inverse “in place,” so that 
after our algorithm is finished the array X[l\ X[2] . . . X[n] will be the inverse 
of the original permutation. Merely setting AT [A" [A]] «- k for 1 < k < n will 
certainly fail, but by considering the cycle structure we can derive the following 
simple algorithm: 

Algorithm I ( Inverse in place). Replace X[1]X[2] . . . X[n], a permutation of 
(1,2, ...,«}, by its inverse. This algorithm is due to Bing-Chao Huang [Inf. 
Proc. Letters 12 (1981), 237-238]. 

11. [Initialize.] Set m «— n, j < 1. 

12. [Next element.] Set i <- X[m]. If i < 0, go to step 15 (the element has 
already been processed). 

13. [Invert one.] (At this point j < 0 and i ~ A'[m]. If m is not the largest ele- 

ment of its cycle, the original permutation had X[-j] = to.) Set X[m] e- j. 
j < m, to i, i <— X[m\. 

14. [End of cycle?] If i > 0, go back to 13 (the cycle has not ended); otherwise 
set i <— j. (In the latter case, the original permutation had X[— j] = to, and 
to is largest in its cycle.) 

15. [Store final value.] Set X[m] < i. (Originally X[—i\ was equal to to.) 

16. [Loop on to.] Decrease to by 1. If to > 0, go back to 12; otherwise the 
algorithm terminates. | 

See Table 3 for an example of this algorithm. The method is based on inversion 
of successive cycles of the permutation, tagging the inverted elements by making 
them negative, afterwards restoring the correct sign. 

Algorithm I resembles parts of Algorithm A, and it very strongly resembles 
the cycle-finding algorithm in Program B (lines 54-68). Thus it is typical 
of a number of algorithms involving rearrangements. When preparing a MIX 
implementation, we find that it is most convenient to keep the value of —i in a 
register instead of i itself: 
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Table 3 

COMPUTING THE INVERSE OF 621543 BY ALGORITHM I 


After step: 

12 

13 

13 

13 

15* 

12 

13 

13 

15 

12 

15 

15 

13 

15 

15 

X[l] 

6 

6 

6 

-3 

-3 

-3 

-3 

-3 

-3 

-3 

-3 

-3 

-3 

-3 

3 

X[2] 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

-4 

2 

2 

m 

1 

1 

-6 

-6 

-6 

-6 

-6 

-6 

-6 

-6 

-6 

6 

6 

6 

6 

X[4] 

5 

5 

5 

5 

5 

5 

5 

-5 

-5 

-5 

5 

5 

5 

5 

5 

A [5] 

4 

4 

4 

4 

4 

4 

-1 

-1 

4 

4 

4 

4 

4 

4 

4 

A [6] 

3 

-1 

-1 

-1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

m 

6 

3 

1 

6 

6 

5 

4 

5 

5 

4 

4 

3 

2 

2 

1 

j 

-1 

-6 

-3 

-1 

-1 

-1 

-5 

-4 

-4 

-4 

-4 

-4 

-2 

-2 

-2 

i 

3 

1 

6 

-1 

-1 

4 

5 

-1 

-4 

-5 

-5 

-6 

-4 

-2 

-3 


Read the columns from left to right. At point *, the cycle (16 3) has been inverted. 


Program I ( Inverse in place), rll = m; rI2 = -i; rI3 = j; and n = N, a symbol 
to be defined when this program is assembled as part of a larger routine. 


01 

INVERT 

ENT1 

N 

1 

11. Initialize, m <— n. 

02 


ENT3 

-1 

1 

j < 1- 

03 

2H 

LD2N 

X,1 

N 

12. Next element, i <— A[ra|. 

04 


J2P 

5F 

N 

To 15 if i < 0. 

05 

3H 

ST3 

X,1 

N 

13. Invert one. Afm] <— 

06 


ENN3 

0,1 

N 

j < m. 

07 


ENN1 

0,2 

N 

m <— i. 

08 


LD2N 

X,1 

N 

i «— A[m], 

09 

4H 

J2N 

3B 

N 

14. End of cycle? To 13 if i > 0. 

10 


ENN2 

0,3 

C 

Otherwise set i <— j. 

11 

5H 

ST2 

X,1 

N 

15. Store final value. A[m] < i. 

12 

6H 

DEC1 

1 

N 

16. Loop on m. 

13 


J1P 

2B 

N 

To 12 if m > 0. | 


The timing for this program is easily worked out in the manner shown earlier; 
every element X [m] is set first to a negative value in step 13 and later to a positive 
value in step 15. The total time comes to (141V + C + 2 )u, where N is the size of 
the array and C is the total number of cycles. The behavior of C in a random 
permutation is analyzed below. 

There is almost always more than one algorithm to do any given task, so 
we would expect that there may be another way to invert a permutation. The 
following ingenious algorithm is due to J. Boothroyd: 

Algorithm J ( Inverse in place). This algorithm has the same effect as Algo- 
rithm I but uses a different method. 

Jl. [Negate all.] Set A[fc] < A[fc], for 1 < k < n. Also set m <— n. 

J2. [Initialize j.] Set j <— m. 

J3. [Find negative entry.] Set i <— X[j\. If i > 0, set j <— i and repeat this step. 
J4. [Invert.] Set X[j] <- X[-i), X[-i] <- m. 

J5. [Loop on m.] Decrease m by 1; if m > 0, go back to J2. Otherwise the 
algorithm terminates. | 
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Table 4 


COMPUTING THE INVERSE OF 62X543 BY ALGORITHM J 


After step: 

32 

J3 

J5 

J3 

J5 

J3 

J5 

J3 

J5 

J3 

J5 

J3 

J5 

X[l] 

-6 

-6 

-6 

-6 

-6 

-6 

-6 

-6 

3 

3 

3 

3 

3 

A [2] 

-2 

-2 

-2 

-2 

-2 

-2 

-2 

-2 

-2 

-2 

2 

2 

2 

A[3] 

-1 

-1 

6 

6 

6 

6 

6 

6 

6 

6 

6 

6 

6 

A[4] 

-5 

-5 

-5 

-5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

X[5] 

-4 

-4 

-4 

-4 

-5 

-5 

4 

4 

4 

4 

4 

4 

4 

A [6] 

-3 

-3 

-1 

-1 

-1 

-1 

-1 

-1 

-6 

-6 

-6 

-6 

1 

m 

6 

6 

5 

5 

4 

4 

3 

3 

2 

2 

1 

1 

0 

i 


-3 

-3 

-4 

-4 

-5 

-5 

-1 

-1 

-2 

-2 

-6 

-6 

j 

6 

6 

6 

5 

5 

5 

5 

6 

6 

2 

2 

6 

6 


See Table 4 for an example of Boothroyd’s algorithm. Again the method is 
essentially based on the cycle structure, but this time it is less obvious that the 
algorithm really works! Verification is left to the reader (see exercise 13). 

Program J ( Analogous to Program I). rll = m; rI2 = j ; rI3 = —i. 


01 

INVERT 

ENN1 

N 

1 

Jl. Negate all. 

02 


ST1 

X+N+l, 1(0:0) 

N 

Set sign negative. 

03 


INC1 

1 

N 


04 


JIN 

*-2 

N 

More? 

05 


ENT1 

N 

1 

m t— n. 

06 

2H 

ENN3 

0,1 

N 

J2. Initialize i. i <— m. 

07 


ENN2 

0,3 

A 

j <- i- 

08 


LD3N 

X,2 

A 

J3. Find negative entry. 

09 


J3N 

*-2 

A 

i > 0? 

10 


LDA 

X,3 

N 

J4. Invert. 

11 


STA 

X,2 

N 

A [j\ t— A[—i]. 

12 


ST1 

X,3 

N 

A[— i] X— m. 

13 


DEC1 

1 

N 

J5. Loov on m. 

U 


J1P 

2B 

N 

To J2 if m > 0. | 


To decide how fast this program runs, we need to know the quantity A ; this 
quantity is so interesting and instructive, it has been left as an exercise (see 
exercise 14). 

Although Algorithm J is deucedly clever, analysis shows that Algorithm I is 
definitely superior. In fact, the average running time of Algorithm J turns out 
to be essentially proportional to nlnn, while that of Algorithm I is essentially 
proportional to n. Maybe some day someone will find a use for Algorithm J (or 
some related modification); it is a bit too pretty to be forgotten altogether. 

An unusual correspondence. We have already remarked that the cycle 
notation for a permutation is not unique; the six-element permutation (1 6 3) (4 5) 
may be written (5 4) (3 1 6), etc. It will be useful to consider a canonical form 
for the cyclic notation; the canonical form is unique. To get the canonical form, 
proceed as follows: 

a) Write all singleton cycles explicitly. 
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b) Within each cycle, put the smallest number first. 

c) Order the cycles in decreasing order of the first number in the cycle. 

For example, starting with (3 1 6) (5 4) we would get 

(a): (3 16) (5 4) (2); (b): (1 6 3) (4 5) (2); (c): (4 5) (2) (1 6 3). ( 20 ) 

The important property of this canonical form is that the parentheses may 
be dropped and uniquely reconstructed again. Thus there is only one way to 
insert parentheses in “4 5 2 1 6 3” to get a canonical cycle form: One must insert 
a left parenthesis just before each left-to-right minimum (namely, just before 
each element that is preceded by no smaller elements). 

This insertion and removal of parentheses gives us an unusual one-to-one 
correspondence between the set of all permutations expressed in cycle form 
and the set of all permutations expressed in linear form. For example, the 
permutation 6 2 1 5 4 3 in canonical cycle form is (4 5) (2) ( 1 6 3); remove 
parentheses to get 4 5 2 1 6 3, which in cycle form is (2 5 6 3) (1 4); remove 
parentheses to get 2 5 6 3 1 4, which in cycle form is (3 6 4) (1 2 5); etc. 

This correspondence has numerous applications to the study of permutations 
of different types. For example, let us ask “How many cycles does a permutation 
of n elements have, on the average?” To answer this question we consider the set 
of all n! permutations expressed in canonical form, and drop the parentheses; we 
are left with the set of all n! permutations in some order. Our original question 
is therefore equivalent to, “How many left-to-right minima does a permutation of 
n elements have, on the average?” We have already answered the latter question 
in Section 1.2.10; this was the quantity ( A + 1) in the analysis of Algorithm 
1.2. 10M, for which we found the statistics 

min 1, ave H n , max n, dev yj H n — Hn\ ( 21 ) 

(Actually, we discussed the average number of right-to-left maxima, but that’s 
clearly the same as the number of left-to-right minima.) Furthermore, we proved 
in essence that a permutation of n objects has k left-to-right minima with prob- 
ability [£] /n!; therefore a permutation of n objects has k cycles with probability 

[;]/»'■ 

We can also ask about the average distance between left-to-right minima, 
which becomes equivalent to the average length of a cycle. By ( 21 ), the total 
number of cycles among all the n! permutations is n! H n , since it is n! times the 
average number of cycles. If we pick one of these cycles at random, what is its 
average length? 

Imagine all n! permutations of {1,2,..., ri} written down in cycle notation; 
how many three-cycles are present? To answer this question, let us consider how 
many times a particular three-cycle ( x y z) appears: It clearly appears in exactly 
(n — 3)! of the permutations, since this is the number of ways the remaining 
n — 3 elements may be permuted. Now the number of different possible three- 
cycles (xy z) is n(n — l)(n — 2)/3, since there are n choices for x, ( n — 1) for y, 
( n — 2) for z, and among these n{n — l)(n — 2) choices each different three-cycle 
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has appeared in three forms ( xyz ), ( yzx ), ( zxy ). Therefore the total number 
of three-cycles among all n! permutations is n(n — l)(n — 2)/3 times (n — 3)!, 
namely n!/3. Similarly, the total number of ?n-cycles is n\/m, for 1 < m < n. 
(This provides another simple proof of the fact that the total number of cycles 
is n! H n : hence the average number of cycles in a random permutation is H n , 
as we already knew.) Exercise 17 shows that the average length of a randomly 
chosen cycle is n/H n , if we consider the n! H n cycles to be equally probable; but 
if we choose an element at random in a random permutation, the average length 
of the cycle containing that element is somewhat greater than n/H n . 

To complete our analyses of Algorithms A and B, we would like to know 
the average number of singleton cycles in a random permutation. This is an 
interesting problem. Suppose we write down the n! permutations, listing first 
those with no singleton cycles, then those with just one, etc.; for example, if 
n = 4, 


no fixed elements: 

2143 

2341 

2413 

3142 

3412 

3421 

4123 

4312 4321 

one fixed element: 

1342 

1423 

3241 

4213 

2431 

4132 

2314 

3124 

two fixed elements: 

1243 

1432 

1324 

4231 

3214 

2134 



three fixed elements: 









four fixed elements: 

1234 









(Singleton cycles, which are the elements that remain fixed by a permutation, 
have been specially marked in this list.) Permutations with no fixed elements 
are called derangements; the number of derangements is the number of ways to 
put n letters into n envelopes, getting them all wrong. 

Let P n k be the number of permutations of n objects having exactly k fixed 
elements, so that for example, 

-F40 = 9, P41 = 8, P42 = 6, P43 = 0, P44 = 1. 


An examination of the list above reveals the principal relationship between these 
numbers: We can get all permutations with k fixed elements by first choosing 
the k that are to be fixed (this can be done in (”) ways) and then permuting 
the remaining n — k elements in all P(„_fc)o ways that leave no further elements 
fixed. Hence 


Pnk — 



P(n — k) 0- 


(22) 


We also have the rule that “the whole is the sum of its parts” : 


n ! — Pnn + P n (n- 1 ) + P n (n- 2 ) + P n (n- 3 ) + ’ ' ’ ■ ( 23 ) 

Combining Eqs. (22) and (23) and rewriting the result slightly, we find that 

n! = — + n— + n{n - 1)— + n(n - 1 )(n - 2 )-^- + ■ ■ • , (24) 

an equation that must be true for all positive integers n. This equation has 
already confronted us before — it appears in Section 1 . 2.5 in connection with 
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Stirling’s attempt to generalize the factorial function — and we found a simple 
derivation of its coefficients in Section 1 . 2.6 (Example 5 ). We conclude that 


^mO 

m! 


„ 1 1 
1_ l! + 2! 


• + (-ir-T- 

ml 


( 2 5 ) 


Now let p n k be the probability that a permutation of n objects has exactly 
k singleton cycles. Since p n k = P n k/n\, we have from Eqs. (22) and (25) 


Pnk 


1 

~k\ 


1 1 
V. + 2 ! 


+ (-1 r 


( n — k)\ 


(26) 


The generating function G n (z) = p n 0 + p n iz + Pn2Z 2 + • • • is therefore 


G n {z) — 1 + jy (z - 1) + 


= E (=7) 


0<j<.n 


From this formula it follows that G' n {z ) = G„_i(z), and with the methods of 
Section 1 . 2.10 we obtain the following statistics on the number of singleton cycles: 


(min 0, ave 1, max n, dev 1), if n > 2. (28) 


A somewhat more direct way to count the number of permutations having 
no singleton cycles follows from the principle of inclusion and exclusion, which is 
an important method for many enumeration problems. The general principle of 
inclusion and exclusion may be formulated as follows: We are given N elements, 
and M subsets, Si, S2, . . . , Sm, of these elements; and our goal is to count how 
many of the elements lie in none of the subsets. Let |S| denote the number of 
elements in a set S ; then the desired number of objects in none of the sets Sj is 

N- £ |5,-|+ £ \Sj n S k \ - £ \SiDSj n Sfcl H 

l<j<M 1 <j<k<M 1 <i<j<k<M 

+ (-i) M |Si n ■ • ■ n s M \ ■ ( 29 ) 


(Thus we first subtract the number of elements in Si, ... , Sm from the total 
number, A r ; but this underestimates the desired total. So we add back the 
number of elements that are common to pairs of sets, Sj fl Sfc, for each pair 
Sj and Sfc; this, however, gives an overestimate. So we subtract the elements 
common to triples of sets, etc.) There are several ways to prove this formula, 
and the reader is invited to discover one of them. (See exercise 25 .) 

To count the number of permutations on n elements having no singleton 
cycles, we consider the N = n! permutations and let Sj be the set of permutations 
in which element j forms a singleton cycle. If 1 < ji < j’2 < • ■ • < jk < n, the 
number of elements in Sj l fl Sj 2 fl • ■ • fl Sj k is the number of permutations in 
which j 1, . . . , jfc are singleton cycles, and this is clearly ( n — k)l. Thus formula 
(29) becomes 

»! - (") (» - D! + (l) (n - 2)! - Q) (» - 3)! + - + (-1)" (") 01, 

in agreement with (25). 
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The principle of inclusion and exclusion is due to A. de Moivre [see his 
Doctrine of Chances (London: 1718), 61-63; 3rd ed. (1756, reprinted by Chelsea, 
1957), 110-112], but its significance was not generally appreciated until it was 
popularized and developed further by I. Todhunter in his Algebra (second edition, 
1860), §762, and by W. A. Whitworth in the well-known book Choice and Chance 
(Cambridge: 1867). 

Combinatorial properties of permutations are explored further in Section 5.1. 

EXERCISES 

1. [02] Consider the transformation of {0, 1, 2, 3, 4, 5, 6} that replaces x by 2x mod 7. 
Show that this transformation is a permutation, and write it in cycle form. 

2. [10] The text shows how we might set (a, b , c, d, e, /) <— (c, d, /, 6, e, a) by using a 
series of replacement operations ( x <— y) and one auxiliary variable t. Show how to do 
the job by using a series of exchange operations ( x y) and no auxiliary variables. 

3. [03] Compute the product (I ^ ° c ^ f e) x (I d f t e a)’ an< ^ ex P ress the answer 
in two-line notation. (Compare with Eq. (4).) 

4. [10] Express (a bd){ef) (a c f) (bd) as a product of disjoint cycles. 

► 5. [M10] Equation (3) shows several equivalent ways to express the same permuta- 
tion in cycle form. How many different ways of writing that permutation are possible, 
if all singleton cycles are suppressed? 

6. [M28] What changes are made to the timing of Program A if we remove the 
assumption that all blank words occur at the extreme right? 

7. [10] If Program A is presented with the input (6), what are the quantities X , Y, M, 
N, U, and V of (19)? What is the time required by Program A, excluding input-output? 

► 8. [23] Would it be feasible to modify Algorithm B to go from left to right instead 
of from right to left through the input? 

9. [10] Both Programs A and B accept the same input and give the answer in 
essentially the same form. Is the output exactly the same under both programs? 

► 10. [M28] Examine the timing characteristics of Program B, namely, the quantities 
A, B, . . . , Z shown there; express the total time in terms of the quantities X, Y, M, N, 
U, V defined in (19), and of F. Compare the total time for Program B with the total 
time for Program A on the input (6), as computed in exercise 7. 

11. [15] Find a simple rule for writing tx~ in cycle form, if the permutation n is given 
in cycle form. 

12. [M2 7] (Transposing a rectangular matrix.) Suppose an m x n matrix (a<j), m / n, 
is stored in memory in a fashion like that of exercise 1.3.2-10, so that the value of a;j 
appears in location L + n(i — 1) + (j — 1), where L is the location of on. The problem is 
to find a way to transpose this matrix, obtaining annxra matrix (&ij), where bij = aji 
is stored in location L + m(i — 1) + ( j — 1). Thus the matrix is to be transposed “on 
itself.” (a) Show that the transposition transformation moves the value that appears 
in cell L + x to cell L + (mx mod N), for all x in the range 0 < x < N = mn — 1. 
(b) Discuss methods for doing this transposition by computer. 

► 13. [M24] Prove that Algorithm J is valid. 

► 14. [M3 4] Find the average value of the quantity A in the timing of Algorithm J. 

15. [M12] Is there a permutation that represents exactly the same transformation 
both in the canonical cycle form without parentheses and in the linear form? 
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16. [Ml 5] Start with the permutation 1324 in linear notation; convert it to canonical 
cycle form and then remove the parentheses; repeat this process until arriving at the 
original permutation. What permutations occur during this process? 

17. [M24] (a) The text demonstrates that there are n\H n cycles altogether, among 
all the permutations on n elements. If these cycles (including singleton cycles) are 
individually written on n! H n slips of paper, and if one of these slips of paper is chosen 
at random, what is the average length of the cycle that is thereby picked? (b) If we write 
the n! permutations on n! slips of paper, and if we choose a number k at random and 
also choose one of the slips of paper, what is the probability that the cycle containing 
k on that slip is an m-cycle? What is the average length of the cycle containing kl 

► 18. [M27] What is p n km, the probability that a permutation of n objects has exactly 
k cycles of length m ? What is the corresponding generating function G„m{z)7 What 
is the average number of m-cycles and what is the standard deviation? (The text 
considers only the case m = 1 .) 

19. [HM21] Show that, in the notation of Eq. ( 25 ), the number P n 0 of derangements 
is exactly equal to n!/e rounded to the nearest integer, for all n > 1 . 

20. [M20] Given that all singleton cycles are written out explicitly, how many different 
ways are there to write the cycle notation of a permutation that has ai one-cycles, 02 
two-cycles, . . . ? (See exercise 5.) 

21. [M22] What is the probability P(n ; cm, 02 , • • • ) that a permutation of n objects 
has exactly ai one-cycles, a 2 two-cycles, etc.? 

► 22. [HM34] (The following approach, due to L. Shepp and S. P. Lloyd, gives a con- 
venient and powerful method for solving problems related to the cycle structure of 
random permutations.) Instead of regarding the number, n, of objects as fixed, and 
the permutation variable, let us assume instead that we independently choose the 
quantities ai, < 22 , Q 3 , . . . appearing in exercises 20 and 21 according to some probability 
distribution. Let w be any real number between 0 and 1. 

a) Suppose that we choose the random variables ai, 02 , < 23 , . . . according to the rule 
that “the probability that a m = k is f(w,m,k),” for some function f(w,m,k). 
Determine the value of f(w,m,k) so that the following two conditions hold: 
(i) J2k>o f ( w ’ m > k) — L for 0 < w < 1 and m > 1 ; (ii) the probability that 
ai + 2a 2 + 3 q 3 + ••• = n and that ai = k\, a 2 = fc 2 , 03 = & 3 , equals 
(1 — w)w n P(n\ k\, fc 2 , & 3 , . . . ), where P(n; k\, k 2 , k $, . . . ) is defined in exercise 21. 

b) A permutation whose cycle structure is an, a 2 , a.%, ... clearly permutes exactly 
ai +2a 2 + 3 a 3 + • • • objects. Show that if the a’s are randomly chosen according to 

the probability distribution in part (a), the probability that ai + 2 a 2 + 3 a 3 -| = n 

is (1 — w)w n ; the probability that ai + 2 a 2 + 3 a 3 + • • • is infinite is zero. 

c) Let 4>{a\, a 2 , . . . ) be any function of the infinitely many numbers an, a 2 , .... 
Show that if the a’s are chosen according to the probability distribution in (a), 
the average value of fi is (1 — w) Y2 n >o wn, t>n\ here fi n denotes the average value 
of fi taken over all permutations of n objects, where the variable aj represents 
the number of j-cycles of a permutation. [For example, if fi(ai, a 2 , . . . ) = an, the 
value of fin is the average number of singleton cycles in a random permutation of 
n objects; we showed in ( 28 ) that fi n = 1 for all n.\ 

d) Use this method to find the average number of cycles of even length in a random 
permutation of n objects. 

e) Use this method to solve exercise 18. 
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23. [HM42] (Golomb, Shepp, Lloyd.) If l n denotes the average length of the longest 
cycle in a permutation of n objects, show that l n « An + |A, where A ~ 0.62433 is a 
constant. Prove in fact that lim n _ >00 (Z Tl — An — |A) = 0. 

24. [M41] Find the variance of the quantity A that enters into the timing of Algo- 
rithm J. (See exercise 14.) 

25. [M22] Prove Eq. (29). 

► 26. [M2 4} Extend the principle of inclusion and exclusion to obtain a formula for 
the number of elements that are in exactly r of the subsets Si, S 2 , ■ ■ ■ , Sm. (The text 
considers only the case r = 0.) 

27. [M20] Use the principle of inclusion and exclusion to count the number of integers 
n in the range 0 < n < ami m2 .. .nit that are not divisible by any of mi, m 2 , . . . ,m t . 
Here mi, m2, . . . , rat, and a are positive integers, with mj _L m*, when j / k. 

28. [ M21 ] (I. Kaplansky.) If the “Josephus permutation” defined in exercise 1.3.2- 
22 is expressed in cycle form, we obtain (1 5 3 6 8 2 4)(7) when n = 8 and m = 4. 
Show that this permutation in the general case is the product (n n— 1 ... 2 l) m_1 x 
(nn-1 ... 2) m-1 ...(nn-l) m - 1 . 

29. [M25] Prove that the cycle form of the Josephus permutation when m = 2 can be 
obtained by first expressing the “doubling” permutation of {1,2 ,..., 2 n}, which takes 
j into (2 j) mod (2 n + 1), in cycle form, then reversing left and right and erasing all 
the numbers greater than n. For example, when n — 11 the doubling permutation 
is (1 2 4 8 16 9 18 13 3 6 12) (5 10 20 17 11 22 21 19 15 7 14) and the Josephus 
permutation is (7 11 10 5)(6 3 9 8 4 2 1). 

30. [M24 ] Use exercise 29 to show that the fixed elements of the Josephus permutation 
when m = 2 are precisely the numbers (2 d — l)(2n + l)/(2 d+1 — 1) for all positive 
integers d such that this is an integer. 

31. [ HM33 ) Generalizing exercises 29 and 30, prove that the fcth man to be executed, 
for general m and n, is in position x, where x may be computed as follows: Set 
x 4— km; then, if x > n, repeatedly set x 4— \_(m(x — n) ~ 1 )/{™ - 1 ) J until x < 11 . 
Consequently the average number of fixed elements, for 1 < n < N and fixed m as 
N — » 00, approaches J2k> i( m — l)/(m fe+1 — (m — 1)) . [Since this value lies between 
(m — 1 )/m and 1, the Josephus permutations have slightly fewer fixed elements than 
random ones do.] 

32. [M25] (a) Prove that any permutation 7r = 7ri7T2 . . . 7T2m+i of the form 

n = (2 3) e2 (4 5) e4 ... (2m 2rn+l) e2m (l 2) ei (3 4) es . . . (2m- 1 2 m) 62 "- 1 , 


where each e/b is 0 or 1, has \nk — k\ < 2 for 1 < k < 2m + 1. 

(b) Given any permutation p of {1,2, ...,n}, construct a permutation n of the 
stated form such that pn is a single cycle. Thus every permutation is “near” a cycle. 


33. [M33] If m = 2 2 and n = 2 2l+1 , show how to construct sequences of permutations 
(atji, aj2, ■ •*>0(jn',l3ji,l3j2, ■ ■ ■ ,Pj n ) for 0 < j < m with the following “orthogonality” 
property: 


• • • C * infdjn 


(12345), if i = j; 

0, if *#.?'• 


Each ctjk and pjk should be a permutation of {1,2,3, 4,5}. 
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► 34. [M25] ( Transposing blocks of data.) One of the most common permutations 
needed in practice is the change from a/3 to [3a. where a and [3 are substrings of an array. 
In other words, if xqx\ . . . x m -i = a and XmXm+i . . . x m +„-i = /3, we want to change 
the array xoxi . . . x m +n- 1 = a/3 to the array x m Xm + i • • • x m + n -iXoXi . . . x m -i = /3a; 
each element Xk should be replaced by x p (k) for 0 < k < m + n, where p(k) = 
(k + m) mod (m + n). Show that every such “cyclic-shift” permutation has a simple 
cycle structure, and exploit that structure to devise a simple algorithm for the desired 
rearrangement. 

35. [M30] Continuing the previous exercise, let xqXi . . . xt+ m +n-i = ct/3^ where a, 
/ 3 , and 7 are strings of respective lengths l, m, and n, and suppose that we want to 
change a/3"f to 7 /3a. Show that the corresponding permutation has a convenient cycle 
structure that leads to an efficient algorithm. [Exercise 34 considered the special case 
m = 0.] Hint: Consider changing (a/ 3 )( 7 / 3 ) to ( 7 / 3 )(a/ 3 ). 

36. [27] Write a MIX subroutine for the algorithm in the answer to exercise 35, and 
analyze its running time. Compare it with the simpler method that goes from a/3j to 
[a/3 r )) R = 'y R /3 R a R to 7 / 3 a, where a R denotes the left-right reversal of the string a. 
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1.4. SOME FUNDAMENTAL PROGRAMMING TECHNIQUES 
1.4.1. Subroutines 

When a certain task is to be performed at several different places in a program, 
it is usually undesirable to repeat the coding in each place. To avoid this 
situation, the coding (called a subroutine) can be put into one place only, and a 
few extra instructions can be added to restart the outer program properly after 
the subroutine is finished. Transfer of control between subroutines and main 
programs is called subroutine linkage. 

Each machine has its own peculiar manner for achieving efficient subroutine 
linkage, usually involving special instructions. In MIX, the J-register is used for 
this purpose; our discussion will be based on MIX machine language, but similar 
remarks will apply to subroutine linkage on other computers. 

Subroutines are used to save space in a program; they do not save any time, 
other than the time implicitly saved by occupying less space — for example, less 
time to load the program, or fewer passes necessary in the program, or better use 
of high-speed memory on machines with several grades of memory. The extra 
time taken to enter and leave a subroutine is usually negligible. 

Subroutines have several other advantages. They make it easier to visualize 
the structure of a large and complex program; they form a logical segmentation 
of the entire problem, and this usually makes debugging of the program easier. 
Many subroutines have additional value because they can be used by people 
other than the programmer of the subroutine. 

Most computer installations have built up a large library of useful sub- 
routines, and such a library greatly facilitates the programming of standard 
computer applications that arise. A programmer should not think of this as the 
only purpose of subroutines, however; subroutines should not always be regarded 
as general-purpose programs to be used by the community. Special-purpose 
subroutines are just as important, even when they are intended to appear in 
only one program. Section 1.4.3. 1 contains several typical examples. 

The simplest subroutines are those that have only one entrance and one exit, 
such as the MAXIMUM subroutine we have already considered (see Section 1.3.2, 
Program M). For reference, we will recopy that program here, changing it so 
that a fixed number of cells, 100, is searched for the maximum: 


* MAXIMUM OF 

X [1 . . 

100] 

MAX100 

STJ 

EXIT 

Subroutine linkage 


ENT3 

100 

Ml. Initialize. 


JMP 

2F 


1H 

CMP A 

X,3 

M3. Compare. 


JGE 

*+3 


2H 

ENT2 

0,3 

M4. Chanse m. 


LDA 

X , 3 

New maximum found 


DEC3 

1 

M5. Decrease k. 


J3P 

IB 

M2. All tested? 

EXIT 

JMP 

* 

Return to main program. | 
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In a larger program containing this coding as a subroutine, the single instruction 
“JMP MAX100” would cause register A to be set to the current maximum value 
of locations X + 1 through X + 100, and the position of the maximum would 
appear in rI2. Subroutine linkage in this case is achieved by the instructions 
“MAX100 STJ EXIT” and, later, “EXIT JMP Because of the way the J-register 
operates, the exit instruction will then jump to the location following the place 
where the original reference to MAX100 was made. 

AX Newer computers, such as the machine MMIX that is destined to replace MIX , 
jl have better ways to remember return addresses. The main difference is that 
program instructions are no longer modified in memory; the relevant information 
is kept in registers or in a special array, not within the program itself. (See 
exercise 7.) The next edition of this book will adopt the modern view, but for 
now we will stick to the old-time practice of self-modifying code. 

It is not hard to obtain quantitative statements about the amount of code 
saved and the amount of time lost when subroutines are used. Suppose that 
a piece of coding requires k locations and that it appears in m places in the 
program. Rewriting this as a subroutine, we need an extra instruction STJ and 
an exit line for the subroutine, plus a single JMP instruction in each of the m 
places where the subroutine is called. This gives a total of m + k + 2 locations, 
rather than mk, so the amount saved is 

(to — 1) (k — 1) — 3. ( 2 ) 

If k is 1 or to is 1 we cannot possibly save any space by using subroutines; this, 
of course, is obvious. If k is 2, m must be greater than 4 in order to gain, etc. 

The amount of time lost is the time taken for the extra JMP, STJ, and JMP 
instructions, which are not present if the subroutine is not used; therefore if the 
subroutine is used t times during a run of the program, 4 1 extra cycles of time 
are required. 

These estimates must be taken with a grain of salt, because they were given 
for an idealized situation. Many subroutines cannot be called simply with a 
single JMP instruction. Furthermore, if the coding is repeated in many parts of a 
program, without using a subroutine approach, the coding for each part can be 
customized to take advantage of special characteristics of the particular part of 
the program in which it lies. With a subroutine, on the other hand, the coding 
must be written for the most general case, not a specific case, and this will often 
add several additional instructions. 

When a subroutine is written to handle a general case, it is expressed in 
terms of parameters. Parameters are values that govern the subroutine’s actions; 
they are subject to change from one call of the subroutine to another. 

The coding in the outside program that transfers control to the subroutine 
and gets it properly started is known as the calling sequence. Particular values 
of parameters, supplied when the subroutine is called, are known as arguments. 
With our MAX 100 subroutine, the calling sequence is simply “JMP MAX 100”, but 
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a longer calling sequence is generally necessary when arguments must be sup- 
plied. For example, Program 1.3.2M is a generalization of MAX100 that finds the 
maximum of the first n elements of the table. The parameter n appears in index 
register 1, and its calling sequence 

LD1 =n= ENT1 n 

or 

JMP MAXIMUM JMP MAXIMUM 

involves two steps. 

If the calling sequence takes c memory locations, formula (2) for the amount 
of space saved changes to 


(m — 1) (fc — c) — constant 


(3) 


and the time lost for subroutine linkage is slightly increased. 

A further correction to the formulas above can be necessary because certain 
registers might need to be saved and restored. For example, in the MAX 100 
subroutine, we must remember that by writing “JMP MAX100” we are not only 
getting the maximum value in register A and its position in register 12; we are 
also setting register 13 to zero. A subroutine may destroy register contents, and 
this must be kept in mind. In order to prevent MAX100 from changing the setting 
of rI3, it would be necessary to include additional instructions. The shortest and 
fastest way to do this with MIX would be to insert the instruction “ST3 3F(0: 2)” 
just after MAX100 and then “3H ENT3 *” just before EXIT. The net cost would 
be an extra two lines of code, plus three machine cycles on every call of the 
subroutine. 

A subroutine may be regarded as an extension of the computer’s machine 
language. With the MAX100 subroutine in memory, we now have a single in- 
struction (namely, “JMP MAX100”) that is a maximum-finder. It is important 
to define the effect of each subroutine just as carefully as the machine language 
operators themselves have been defined; a programmer should therefore be sure 
to write down the characteristics of each subroutine, even though nobody else 
will be making use of the routine or its specification. In the case of MAXIMUM as 
given in Section 1.3.2, the characteristics are as follows: 


Calling sequence: JMP MAXIMUM. 

Entry conditions: rll = n; assume that n > 1. 

Exit conditions: rA = max CONTENTS (X + A:) = CONTENTS (X + rI2): 

l<fc<n 

rI3 = 0; rJ and Cl are also affected. 


} (4) 


(We will customarily omit mention of the fact that register J and the compar- 
ison indicator are affected by a subroutine; it has been mentioned here only 
for completeness.) Note that rX and rll are unaffected by the action of the 
subroutine, for otherwise these registers would have been mentioned in the exit 
conditions. A specification should also mention all memory locations external 
to the subroutine that might be affected; in this case the specification allows us 
to conclude that nothing has been stored, since (4) doesn’t say anything about 
changes to memory. 


1.4.1 


SUBROUTINES 189 


Now let’s consider multiple entrances to subroutines. Suppose we have a 
program that requires the general subroutine MAXIMUM, but it usually wants to use 
the special case MAX100 in which n — 100. The two can be combined as follows: 

MAX100 ENT3 100 First entrance 

MAXN STJ EXIT Second entrance 

JMP 2F Continue as in (l). ( 5 ) 

EXIT JMP * Return to main program. | 

Subroutine ( 5 ) is essentially the same as ( 1 ), with the first two instructions 
interchanged; we have used the fact that “ENT3” does not change the setting of 
the J-register. If we wanted to add a third entrance, MAX50, to this subroutine, 
we could insert the code 

MAX50 ENT3 50 

JSJ MAXN ’ 

at the beginning. (Recall that “JSJ” means jump without changing register J.) 

When the number of parameters is small, it is often desirable to transmit 
them to a subroutine either by having them in convenient registers (as we have 
used rI3 to hold the parameter n in MAXN and as we used rll to hold the parameter 
n in MAXIMUM), or by storing them in fixed memory cells. 

Another convenient way to supply arguments is simply to list them after 
the JMP instruction; the subroutine can refer to its parameters because it knows 
the J-register setting. For example, if we wanted to make the calling sequence 
for MAXN be 


JMP MAXN 
CON n 

then the subroutine could be written as follows: 


MAXN STJ 

*+l 


ENT1 

* 

rll <- rJ. 

LD3 

0,1 

rI3 <— n. 

JMP 

2F 

Continue as in ( 1 ) 

J3P 

IB 


JMP 

1,1 

Return. | 


(7) 


( 8 ) 


On machines like System/360, for which linkage is ordinarily done by putting the 
exit location in an index register, a convention like this is particularly convenient. 
It is also useful when a subroutine needs many arguments, or when a program 
has been written by a compiler. The technique of multiple entrances that we 
used above often fails in this case, however. We could “fake it” by writing 


MAX100 STJ IF 

JMP MAXN 
CON 100 

1H JMP * | 


but this is not as attractive as ( 5 ). 
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A technique similar to that of listing arguments after the jump is normally 
used for subroutines with multiple exits. Multiple exit means that we want the 
subroutine to return to one of several different locations, depending on conditions 
detected by the subroutine. In the strictest sense, the location to which a 
subroutine exits is a parameter; so if there are several places to which it might 
exit, depending on the circumstances, they should be supplied as arguments. 
Om final example of the maximum subroutine will have two entrances and 
two exits. The calling sequence is: 

For general n For n = 100 

ENT3 n 

JMP MAXN JMP MAX100 

Exit here if max < 0 or max > rX. Exit here if max < 0 or max > rX. 

Exit here if 0 < max < rX. Exit here if 0 < max < rX. 

(In other words, exit is made to the location two past the jump when the 
maximum value is positive and less than the contents of register X.) The 
subroutine for these conditions is easily written: 


MAX100 

ENT3 

100 

Entrance for n = 100 

MAXN 

STJ 

EXIT 

Entrance for general n 


JMP 

2F 

Continue as in (i). 


J3P 

IB 



JANP 

EXIT 

Take normal exit if the max is < 0. 


STX 

TEMP 



CMPA 

TEMP 



JGE 

EXIT 

Take normal exit if the max is > rX. 


ENT3 

1 

Otherwise take the second exit. 

EXIT 

JMP 

*.3 

Return to proper place. | 


Subroutines may call on other subroutines; in complicated programs it is not 
unusual to have subroutine calls nested more than five deep. The only restriction 
that must be followed when using linkage as described here is that no subroutine 
may call on any other subroutine that is (directly or indirectly) calling on it. For 
example, consider the following scenario: 


[Main program] 

[Subroutine A] 

A STJ EXITA 

[Subroutine B] 

B STJ EXITB 

[Subroutine C] 

C STJ EXITC 

JMP A 

JMP B 

JMP C 

JMP A 


EXITA JMP * 

EXITB JMP * 

EXITC JMP * (io 


If the main program calls on A, which calls B, which calls C, and then C 
calls on A, the address in EXITA referring to the main program is destroyed, 
and there is no way to return to that program. A similar remark applies to all 
temporary storage cells and registers used by each subroutine. It is not difficult 
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to devise subroutine linkage conventions that will handle such recursive situations 
properly; Chapter 8 considers recursion in detail. 

We conclude this section by discussing briefly how we might go about writing 
a complex and lengthy program. How can we decide what kind of subroutines 
we will need, and what calling sequences should be used? One successful way to 
determine this is to use an iterative procedure: 

Step 0 (Initial idea). First we decide vaguely upon the general plan of 
attack that the program will use. 

Step 1 (A rough sketch of the program). We start now by writing the “outer 
levels” of the program, in any convenient language. A somewhat systematic way 
to go about this has been described very nicely by E. W. Dijkstra, Structured 
Programming (Academic Press, 1972), Chapter 1, and by N. Wirth, CACM 
14 (1971), 221-227. We may begin by breaking the whole program into a 
small number of pieces, which might be thought of temporarily as subroutines, 
although they are called only once. These pieces are successively refined into 
smaller and smaller parts, having correspondingly simpler jobs to do. Whenever 
some computational task arises that seems likely to occur elsewhere or that has 
already occurred elsewhere, we define a subroutine (a real one) to do that job. 
We do not write the subroutine at this point; we continue writing the main 
program, assuming that the subroutine has performed its task. Finally, when 
the main program has been sketched, we tackle the subroutines in turn, trying 
to take the most complex subroutines first and then their sub-subroutines, etc. 
In this manner we will come up with a list of subroutines. The actual function 
of each subroutine has probably already changed several times, so that the first 
parts of our sketch will by now be incorrect; but that is no problem, it is merely 
a sketch. For each subroutine we now have a reasonably good idea about how it 
will be called and how general-purpose it should be. It usually pays to extend 
the generality of each subroutine a little. 

Step 2 (First working program). This step goes in the opposite direction 
from step 1. We now write in computer language, say MIXAL or PL/MIX or 
a higher-level language; we start this time with the lowest level subroutines, 
and do the main program last. As far as possible, we try never to write any 
instructions that call a subroutine before the subroutine itself has been coded. 
(In step 1, we tried the opposite, never considering a subroutine until all of its 
calls had been written.) 

As more and more subroutines are written during this process, our con- 
fidence gradually grows, since we are continually extending the power of the 
machine we are programming. After an individual subroutine is coded, we should 
immediately prepare a complete description of what it does, and what its calling 
sequences are, as in ( 4 ). It is also important not to overlay temporary storage 
cells; it may very well be disastrous if every subroutine refers to location TEMP, 
although when preparing the sketch in step 1 , it was convenient not to worry 
about such problems. An obvious way to overcome overlay worries is to have 
each subroutine use only its own temporary storage, but if this is too wasteful 
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of space, another scheme that does fairly well is to name the cells TEMPI, TEMP2. 
etc.; the numbering within a subroutine starts with TEMPj, where j is one higher 
than the greatest number used by any of the sub- subroutines of this subroutine. 

Step 3 (Reexamination). The result of step 2 should be very nearly a 
working program, but it may be possible to improve on it. A good way is to 
reverse direction again, studying for each subroutine all of the calls made on it. 
It may well be that the subroutine should be enlarged to do some of the more 
common things that are always done by the outside routine just before or after 
it uses the subroutine. Perhaps several subroutines should be merged into one; 
or perhaps a subroutine is called only once and should not be a subroutine at 
all. (Perhaps a subroutine is never called and can be dispensed with entirely.) 

At this point, it is often a good idea to scrap everything and start over 
again at step 1! This is not intended to be a facetious remark; the time spent in 
getting this far has not been wasted, for we have learned a great deal about the 
problem. With hindsight, we will probably have discovered several improvements 
that could be made to the program’s overall organization. There’s no reason to 
be afraid to go back to step 1 — it will be much easier to go through steps 2 
and 3 again, now that a similar program has been done already. Moreover, we 
will quite probably save as much debugging time later on as it will take to rewrite 
everything. Some of the best computer programs ever written owe much of their 
success to the fact that all the work was unintentionally lost, at about this stage, 
and the authors had to begin again. 

On the other hand, there is probably never a point when a complex computer 
program cannot be improved somehow, so steps 1 and 2 should not be repeated 
indefinitely. When significant improvement can clearly be made, it is well worth 
the additional time required to start over, but eventually a point of diminishing 
returns is reached. 

Step 4 (Debugging). After a final polishing of the program, including 
perhaps the allocation of storage and other last-minute details, it is time to 
look at it in still another direction from the three that were used in steps 1. 
2, and 3 — now we study the program in the order in which the computer will 
perform it. This may be done by hand or, of course, by machine. The author 
has found it quite helpful at this point to make use of system routines that trace 
each instruction the first two times it is executed; it is important to rethink the 
ideas underlying the program and to check that everything is actually taking 
place as expected. 

Debugging is an art that needs much further study, and the way to approach 
it is highly dependent on the facilities available at each computer installation. 
A good start towards effective debugging is often the preparation of appropriate 
test data. The most effective debugging techniques seem to be those that are 
designed and built into the program itself — many of today’s best programmers 
will devote nearly half of their programs to facilitating the debugging process 
in the other half, the first half, which usually consists of fairly straightforward 
routines that display relevant information in a readable format, will eventually 
be thrown away, but the net result is a surprising gain in productivity. 
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Another good debugging practice is to keep a record of every mistake made. 
Even though this will probably be quite embarrassing, such information is in- 
valuable to anyone doing research on the debugging problem, and it will also 
help you learn how to reduce the number of future errors. 

Note: The author wrote most of the preceding comments in 1964, after he 
had successfully completed several medium-sized software projects but before 
he had developed a mature programming style. Later, during the 1980s, he 
learned that an additional technique, called structured documentation or literate 
programming, is probably even more important. A summary of his current beliefs 
about the best way to write programs of all kinds appears in the book Literate 
Programming (Cambridge Univ. Press, first published in 1992). Incidentally, 
Chapter 11 of that book contains a detailed record of all bugs removed from the 
TgJC program during the period 1978-1991. 

Up to a point it is better to let the snags [bugs] be there 
than to spend such time in design that there are none 
(how many decades would this course take?). 

— A. M. TURING, Proposals for ACE (1945) 


EXERCISES 

1. [10] State the characteristics of subroutine (5), just as (4) gives the characteristics 
of Subroutine 1.3.2M. 

2. [10] Suggest code to substitute for (6) without using the JSJ instruction. 

3. [M15] Complete the information in (4) by stating precisely what happens to 
register J and the comparison indicator as a result of the subroutine; state also what 
happens if register II is not positive. 

► 4. [21] Write a subroutine that generalizes MAXN by finding the maximum value of 
X [a] , X [a + r] , X [a + 2 r] , . . . , X [n] , where r and n are parameters and a is the smallest 
positive number with a = n (modulo r), namely a = 1 + (n — 1) mod r. Give a special 
entrance for the case r = 1. List the characteristics of your subroutine, as in (4). 

5. [21] Suppose MIX did not have a J-register. Invent a means for subroutine linkage 
that does not use register ,J, and give an example of your invention by writing a MAX100 
subroutine effectively equivalent to (1). State the characteristics of this subroutine in 
a fashion similar to (4). (Retain Mix’s conventions of self-modifying code.) 

► 6. [26] Suppose MIX did not have a MOVE operator. Write a subroutine entitled MOVE 
such that the calling sequence “JMP MOVE; NOP A, 1(F)” has an effect just the same as 
■’MOVE A, 1(F)” if the latter were admissible. The only differences should be the effect 
on register J and the fact that a subroutine naturally consumes more time and space 
than a hardware instruction does. 

► 7. [20] Why is self-modifying code now frowned on? 

1.4.2. Coroutines 

Subroutines are special cases of more general program components, called co- 
routines. In contrast to the unsymmetric relationship between a main routine 
and a subroutine, there is complete symmetry between coroutines, which call on 
each other. 
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To understand the coroutine concept, let us consider another way of thinking 
about subroutines. The viewpoint adopted in the previous section was that a 
subroutine merely was an extension of the computer hardware, introduced to 
save lines of coding. This may be true, but another point of view is possible: 
We may consider the main program and the subroutine as a team of programs, 
each member of the team having a certain job to do. The main program, in 
the course of doing its job, will activate the subprogram; the subprogram will 
perform its own function and then activate the main program. We might stretch 
our imagination to believe that, from the subroutine’s point of view, when it 
exits it is calling the main routine; the main routine continues to perform its 
duty, then “exits” to the subroutine. The subroutine acts, then calls the main 
routine again. 

This somewhat far-fetched philosophy actually takes place with coroutines, 
for which it is impossible to distinguish which is a subroutine of the other. 
Suppose we have coroutines A and B; when programming A, we may think of B as 
our subroutine, but when programming B, we may think of A as our subroutine. 
That is, in coroutine A, the instruction “JMP B” is used to activate coroutine B. 
In coroutine B the instruction “JMP A” is used to activate coroutine A again. 
Whenever a coroutine is activated, it resumes execution of its program at the 
point where the action was last suspended. 

The coroutines A and B might, for example, be two programs that play chess. 
We can combine them so that they will play against each other. 

With MIX, such linkage between coroutines A and B is done by including the 
following four instructions in the program: 

A STJ BX B STJ AX 

AX JMP A1 BX JMP B1 

This requires four machine cycles for transfer of control each way. Initially AX and 
BX are set to jump to the starting places of each coroutine, A1 and Bl. Suppose we 
start up coroutine A first, at location Al. When it executes “JMP B” from location 
A2, say, the instruction in location B stores rJ in AX, which then says “JMP A2+1”. 
The instruction in BX gets us to location Bl, and after coroutine B begins its 
execution, it will eventually get to an instruction “JMP A” in location B2, say. We 
store rJ in BX and jump to location A2+1, continuing the execution of coroutine 
A until it again jumps to B, which stores rJ in AX and jumps to B2+1, etc. 

The essential difference between routine-subroutine and coroutine-coroutine 
linkage, as can be seen by studying the example above, is that a subroutine is 
always initiated at its beginning, which is usually a fixed place; the main routine 
or a coroutine is always initiated at the place following where it last terminated. 

Coroutines arise most naturally in practice when they are connected with 
algorithms for input and output. For example, suppose it is the duty of corou- 
tine A to read cards and to perform some transformation on the input, reducing 
it to a sequence of items. Another coroutine, which we will call B, does further 
processing of these items, and prints the answers; B will periodically call for the 
successive input items found by A. Thus, coroutine B jumps to A whenever it 
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wants the next input item, and coroutine A jumps to B whenever an input item 
has been found. The reader may say, “Well, B is the main program and A is 
merely a subroutine for doing the input.” This, however, becomes less true when 
the process A is very complicated; indeed, we can imagine A as the main routine 
and B as a subroutine for doing the output, and the above description remains 
valid. The usefulness of the coroutine idea emerges midway between these two 
extremes, when both A and B are complicated and each one calls the other in 
numerous places. It is rather difficult to find short, simple examples of coroutines 
that illustrate the importance of the idea; the most useful coroutine applications 
are generally quite lengthy. 

In order to study coroutines in action, let us consider a contrived example. 
Suppose we want to write a program that translates one code into another. The 
input code to be translated is a sequence of alphameric characters terminated 
by a period, such as 

A2B5E3426FG0ZYW3210PQ89R . (2) 

This has been punched onto cards; blank columns appearing on these cards are 
to be ignored. The input is to be understood as follows, from left to right: If the 
next character is a digit 0, 1, . . . , 9, say n, it indicates (n + 1) repetitions of the 
following character, whether the following character is a digit or not. A nondigit 
simply denotes itself. The output of our program is to consist of the sequence 
indicated in this manner and separated into groups of three characters each, 
until a period appears; the last group may have fewer than three characters. For 
example, (2) should be translated by our program into 

ABB BEE EEE E44 446 66F GZY W22 220 OPQ 999 999 999 R. (3) 

Note that 3426F does not mean 3427 repetitions of the letter F: it means 4 fours 
and 3 sixes followed by F. If the input sequence is T . ’, the output is simply ‘ . ’, 
not because the first period terminates the output. Our program should 

punch the output onto cards, with sixteen groups of three on each card except 
possibly the last. 

To accomplish this translation, we will write two coroutines and a subrou- 
tine. The subroutine, called NEXTCHAR, is designed to find nonblank characters 
of the input, and to put the next such character into register A: 


01 

* SUBROUTINE FOR CHARACTER 

INPUT 

02 

READER 

EQU 

16 

Unit number of card reader 

03 

INPUT 

ORIG 

*+16 

Place for input cards 

04 

NEXTCHAR 

STJ 

9F 

Entrance to subroutine 

05 


JXNZ 

3F 

Initially rX = 0 

06 

1H 

J6N 

2F 

Initially rI6 = 0 

07 


IN 

INPUT (READER) 

Read next card. 

08 


JBUS 

* (READER) 

Wait for completion. 

09 


ENN6 

16 

Let rI6 point to the first word 

10 

2H 

LDX 

INPUT+16 , 6 

Get the next word of input. 

11 


INC6 

1 

Advance pointer. 
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12 3H ENT A 0 

13 SLAX 1 Next character — » rA. 

14 9H JANZ * Skip blanks. 

15 JMP NEXTCHAR+1 | 

This subroutine has the following characteristics: 

Calling sequence: JMP NEXTCHAR. 

Entry conditions: rX = characters yet to be used; rI6 points to next word, or 
rI6 = 0 indicating that a new card must be read. 

Exit conditions: rA = next nonblank character of input; rX and rI6 are set for 
next entry to NEXTCHAR. 

Our first coroutine, called IN, finds the characters of the input code with 
the proper replication. It begins initially at location INI: 


16 

* FIRST COROUTINE 


17 

2H 

INCA 

30 

Nondigit found 

18 


JMP 

OUT 

Send it to OUT coroutine. 

19 

INI 

JMP 

NEXTCHAR 

Get character. 

20 


DECA 

30 


21 


JAN 

2B 

Is it a letter? 

22 


CMPA 

=10= 


23 


JGE 

2B 

Is it a special character? 

24 


STA 

*+1(0:2) 

Digit n found 

25 


ENT5 

* 

rI5 «— n. 

26 


JMP 

NEXTCHAR 

Get next character. 

27 


JMP 

OUT 

Send it to OUT coroutine. 

28 


DEC5 

1 

Decrease n by 1. 

29 


J5NN 

*-2 

Repeat if necessary. 

30 


JMP 

INI 

Begin new cycle. | 


(Recall that in Mix’s character code, the digits 0-9 have codes 30-39.) This 
coroutine has the following characteristics: 

Calling sequence: JMP IN. 

Exit conditions (when 

jumping to OUT): rA = next character of input with proper replication; rI4 
unchanged from its value at entry. 

Entry conditions 

(upon return): rA, rX, rI5, rI6 should be unchanged from their values 

at the last exit. 

The other coroutine, called OUT, puts the code into three-character groups 


and 

31 

punches the cards. It begins initially at 0UT1: 

* SECOND COROUTINE 

32 

ALF 

Constant used for blanking 

33 

OUTPUT 0RIG *+16 

Buffer area for answers 

34 

PUNCH EQU IT 

Unit number for card punch 

35 

0UT1 ENT4 -16 

Start new output card. 
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36 


ENT1 

OUTPUT 

37 


MOVE 

-1,1(16) 

38 

1H 

JMP 

IN 

39 


STA 

0UTPUT+16,4(1 : 1) 

40 


CMPA 

PERIOD 

41 


JE 

9F 

42 


JMP 

IN 

43 


STA 

OUTPUT+16 ,4 (2 : 2) 

44 


CMPA 

PERIOD 

45 


JE 

9F 

46 


JMP 

IN 

47 


STA 

OUTPUT+16, 4 (3: 3) 

48 


CMPA 

PERIOD 

49 


JE 

9F 

50 


INC4 

1 

51 


J4N 

IB 

52 

9H 

OUT 

OUTPUT (PUNCH) 

53 


JBUS 

* (PUNCH) 

54 


JNE 

0UT1 

55 


HLT 


56 

PERIOD 

ALF 

uuuu • 


Set output area to blanks. 

Get next translated character. 

Store it in the (1:1) field. 

Is it “.”? 

If not, get another character. 

Store it in the (2 : 2) field. 

Is it 

If not, get another character. 

Store it in the (3:3) field. 

Is it 

Move to next word in output buffer. 
End of card? 

If so, punch it. 

Wait for completion. 

Return for more, unless 
was sensed. 

I 


This coroutine has the following characteristics: 


Calling sequence: 

Exit conditions (when 
jumping to IN): 

Entry conditions 
(upon return): 


JMP OUT. 

rA, rX, rI5, rI6 unchanged from their value at entry; rll 
possibly affected; previous character recorded in output. 

rA = next character of input with proper replication; rI4 
unchanged from its value at the last exit. 


To complete the program, we need to write the coroutine linkage (see (i)) 
and to provide the proper initialization. Initialization of coroutines tends to be 
a little tricky, although not really difficult. 


57 

* INITIALIZATION 

AND LINKAGE 

58 

START 

ENT6 

0 

Initialize rI6 for NEXTCHAR. 

59 


ENTX 

0 

Initialize rX for NEXTCHAR. 

60 


JMP 

0UT1 

Start with OUT (see exercise 

61 

OUT 

STJ 

INX 

Coroutine linkage 

62 

OUTX 

JMP 

0UT1 


63 

IN 

STJ 

OUTX 


64 

INX 

JMP 

INI 


65 


END 

START 

1 


This completes the program. The reader should study it carefully, noting in 
particular how each coroutine can be written independently as though the other 
coroutine were its subroutine. 
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The entry and exit conditions for the IN and OUT coroutines mesh perfectly in 
the program above. In general, we would not be so fortunate, and the coroutine 
linkage would also include instructions for loading and storing appropriate regis- 
ters. For example, if OUT would destroy the contents of register A, the coroutine 
linkage would become 


OUT 

STJ 

INX 



STA 

HOLDA 

Store A when leaving IN. 

OUTX 

JMP 

0UT1 

IN 

STJ 

OUTX 



LDA 

HOLDA 

Restore A when leaving OUT 

INX 

JMP 

INI 

1 


There is an important relation between coroutines and multipass algorithms. 
For example, the translation process we have just described could have been done 
in two distinct passes: We could first have done just the IN coroutine, applying 
it to the entire input and writing each character with the proper amount of 
replication onto magnetic tape. After this was finished, we could rewind the 
tape and then do just the OUT coroutine, taking the characters from tape in 
groups of three. This would be called a “two-pass” process. (Intuitively, a 
“pass” denotes a complete scan of the input. This definition is not precise, 
and in many algorithms the number of passes taken is not at all clear; but the 
intuitive concept of “pass” is useful in spite of its vagueness.) 

Figure 22(a) illustrates a four-pass process. Quite often we will find that 
the same process can be done in just one pass, as shown in part (b) of the figure, 
if we substitute four coroutines A, B, C, D for the respective passes A, B, C, D. 
Coroutine A will jump to B when pass A would have written an item of output 
on tape 1; coroutine B will jump to A when pass B would have read an item of 
input from tape 1, and B will jump to C when pass B would have written an item 
of output on tape 2; etc. UNIX i?! users will recognize this as a “pipe,” denoted 
by “PassA | PassB | PassC I PassD”. The programs for passes B, C, and D 
are sometimes referred to as “filters.” 

Conversely, a process done by n coroutines can often be transformed into an 
n-pass process. Due to this correspondence it is worthwhile to compare multipass 
algorithms with one-pass algorithms. 

a) Psychological difference. A multipass algorithm is generally easier to create 
and to understand than a one-pass algorithm for the same problem. Breaking a 
process down into a sequence of small steps that happen one after the other is 
easier to comprehend than an involved process in which many transformations 
take place simultaneously. 

Also, if a very large problem is being tackled and if many people are to 
co-operate in producing a computer program, a multipass algorithm provides a 
natural way to divide up the job. 

These advantages of a multipass algorithm are present in coroutines as well, 
since each coroutine can be written essentially separate from the others, and the 
linkage makes an apparently multipass algorithm into a single-pass process. 
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Fig. 22. Passes: (a) a four-pass algorithm, and (b) a one-pass algorithm. 


b) Time difference. The time required to pack, write, read, and unpack the 
intermediate data that flows between passes (for example, the information on 
tapes in Fig. 22) is avoided in a one-pass algorithm. For this reason, a one-pass 
algorithm will be faster. 

c) Space difference. The one-pass algorithm requires space to hold all the 
programs in memory simultaneously, while a multipass algorithm requires space 
for only one at a time. This requirement may affect the speed, even to a greater 
extent than indicated in statement (b). For example, many computers have a 
limited amount of “fast memory” and a larger amount of slower memory; if each 
pass just barely fits into the fast memory, the result will be considerably faster 
than if we use coroutines in a single pass (since the use of coroutines would 
presumably force most of the program to appear in the slower memory or to be 
repeatedly swapped in and out of fast memory). 

Occasionally there is a need to design algorithms for several computer con- 
figurations at once, some of which have larger memory capacity than others. In 
such cases it is possible to write the program in terms of coroutines, and to let 
the memory size govern the number of passes: Load together as many coroutines 
as feasible, and supply input or output subroutines for the missing links. 

Although this relationship between coroutines and passes is important, we 
should keep in mind that coroutine applications cannot always be split into 
multipass algorithms. If coroutine B gets input from A and also sends back 
crucial information to A, as in the example of chess play mentioned earlier, the 
sequence of actions can’t be converted into pass A followed by pass B. 

Conversely, it is clear that some multipass algorithms cannot be converted 
to coroutines. Some algorithms are inherently multipass; for example, the second 
pass may require cumulative information from the first pass (like the total 
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number of occurrences of a certain word in the input). There is an old joke 
worth noting in this regard: 

Little old lady, riding a bus. “Little boy, can you tell me how to get off 
at Pasadena Street?” 

Little boy. “Just watch me, and get off two stops before I do.” 

(The joke is that the little boy gives a t.wo-pass algorithm.) 

So much for multipass algorithms. We will see further examples of coroutines 
in numerous places throughout this book, for example, as part of the buffering 
schemes in Section 1.4.4. Coroutines also play an important role in discrete 
system simulation; see Section 2.2.5. The important idea of replicated coroutines 
is discussed in Chapter 8, and some interesting applications of this idea may be 
found in Chapter 10. 

EXERCISES 

1. [10] Explain why short, simple examples of coroutines are hard for the author of 
a textbook to Bnd. 

► 2. [20] The program in the text starts up the OUT coroutine first. What would 
happen if IN were the first to be executed — that is, if line 60 were changed from 

“JMP 0UT1” to “JMP INI”? 

3. [20] True or false: The three “CMPA PERIOD” instructions within OUT may all be 
omitted, and the program would still work. (Look carefully.) 

4. [20] Show how coroutine linkage analogous to (i) can be given for real-life com- 
puters you are familiar with. 

5. [15] Suppose both coroutines IN and OUT want the contents of register A to remain 
untouched between exit and entry; in other words, assume that wherever the instruction 
“JMP IN” occurs within OUT, the contents of register A are to be unchanged when control 
returns to the next line, and make a similar assumption about “JMP OUT" within IN. 
What coroutine linkage is needed? (Compare with (4).) 

► 6. [22] Give coroutine linkage analogous to (1) for the case of three coroutines, A. B. 
and C, each of which can jump to either of the other two. (Whenever a coroutine is 
activated, it begins where it last left off.) 

► 7. [30] Write a MIX program that reverses the translation done by the program in the 
text; that is, your program should convert cards punched like (3) into cards punched 
like (2). The output should be as short a string of characters as possible, so that the 
zero before the Z in (2) would not really be produced from (3). 

1.4.3. Interpretive Routines 

In this section we will investigate a common type of computer program, the 
interpretive routine (which will be called interpreter for short). An interpretive 
routine is a computer program that performs the instructions of another pro- 
gram, where the other program is written in some machine-like language. By a 
machine-like language, we mean a way of representing instructions, where the 
instructions typically have operation codes, addresses, etc. (This definition, like 
most definitions of today’s computer terms, is not precise, nor should it be; we 
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cannot draw the line exactly and say just which programs are interpreters and 
which are not.) 

Historically, the first interpreters were built around machine-like languages 
designed specially for simple programming; such languages were easier to use 
than a real machine language. The rise of symbolic languages for programming 
soon eliminated the need for interpretive routines of that kind, but interpreters 
have by no means begun to die out. On the contrary, their use has continued 
to grow, to the extent that an effective use of interpretive routines may be 
regarded as one of the essential characteristics of modern programming. The 
new applications of interpreters are made chiefly for the following reasons: 

a) a machine-like language is able to represent a complicated sequence of deci- 
sions and actions in a compact, efficient manner; and 

b) such a representation provides an excellent way to communicate between 

passes of a multipass process. 

In such cases, special purpose machine- like languages are developed for use in 
a particular program, and programs in those languages are often generated only 
by computers. (Today’s expert programmers are also good machine designers, as 
they not only create an interpretive routine, they also define a virtual machine 
whose language is to be interpreted.) 

The interpretive technique has the further advantage of being relatively 
machine-independent — only the interpreter must be rewritten when changing 
computers. Furthermore, helpful debugging aids can readily be built into an 
interpretive system. 

Examples of interpreters of type (a) appear in several places later in this 
series of books; see, for example, the recursive interpreter in Chapter 8 and the 
“Parsing Machine” in Chapter 10. We typically need to deal with a situation in 
which a great many special cases arise, all similar, but having no really simple 
pattern. 

For example, consider writing an algebraic compiler in which we want to gen- 
erate efficient machine-language instructions that add two quantities together. 
There might be ten classes of quantities (constants, simple variables, temporary 
storage locations, subscripted variables, the contents of an accumulator or index 
register, fixed or floating point, etc.) and the combination of all pairs yields 100 
different cases. A long program would be required to do the proper thing in 
each case. The interpretive solution to this problem is to make up an ad hoc 
language whose “instructions” fit in one byte. Then we simply prepare a table 
of 100 “programs” in this language, where each program ideally fits in a single 
word. The idea is then to pick out the appropriate table entry and to perform 
the program found there. This technique is simple and efficient. 

An example interpreter of type (b) appears in the article “Computer-Drawn 
Flowcharts” by D. E. Knuth, CACM 6 (1963), 555-563. In a multipass program, 
the earlier passes must transmit information to the later passes. This information 
is often transmitted most efficiently in a machine-like language, as a set of 
instructions for the later pass; the later pass is then nothing but a special purpose 
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interpretive routine, and the earlier pass is a special purpose “compiler.” This 
philosophy of multipass operation may be characterized as telling the later pass 
what to do, whenever possible, rather than simply presenting it with a lot of 
facts and asking it to figure out what to do. 

Another example of a type-(b) interpreter occurs in connection with com- 
pilers for special languages. If the language includes many features that are not 
easily done on the machine except by subroutine, the resulting object programs 
will be very long sequences of subroutine calls. This would happen, for ex am ple, 
if the language were concerned primarily with multiple precision arithmetic. In 
such a case the object program would be considerably shorter if it were expressed 
in an interpretive language. See, for example, the book ALGOL 60 Implementa- 
tion, by B. Randell and L. J. Russell (New York: Academic Press, 1964), which 
describes a compiler to translate from ALGOL 60 into an interpretive language, 
and which also describes the interpreter for that language; and see “An ALGOL 
60 Compiler,” by Arthur Evans, Jr., Ann. Rev. Auto. Programming 4 (1964), 
87-124, for examples of interpretive routines used within a compiler. The rise of 
microprogrammed machines and of special-purpose integrated circuit chips has 
made this interpretive approach even more valuable. 

The TgX program, which produced the pages of the book you are now 
reading, converted a file that contained the text of this section into an interpretive 
language called DVI format, designed by D. R. Fuchs in 1979. [See D. E. Knuth, 
Lfi]X: The Program (Reading, Mass.: Addison- Wesley, 1986), Part 31.] The DVI 
file that produced was then processed by an interpreter called dvips. written 
by T. G. Rokicki, and converted to a hie of instructions in another interpretive 
language called PostScript® [Adobe Systems Inc., PostScript Language Reference 
Manual, 2nd edition (Reading, Mass.: Addison- Wesley, 1990)]. The PostScript 
file was sent to the publisher, who sent it to a commercial printer, who used 
a PostScript interpreter to produce printing plates. This three-pass operation 
illustrates interpreters of type (b); TgX itself also includes a small interpreter of 
type (a) to process the so-called ligature and kerning information for characters 
of each font of type [IfeX: The Program, §545], 

There is another way to look at a program written in interpretive language: 
It may be regarded as a series of subroutine calls, one after another. Such a pro- 
gram may in fact be expanded into a long sequence of calls on subroutines, and, 
conversely, such a sequence can usually be packed into a coded form that is read- 
ily interpreted. The advantages of interpretive techniques are the compactness of 
representation, the machine independence, and the increased diagnostic capabil- 
ity. An interpreter can often be written so that the amount of time spent in inter- 
pretation of the code itself and branching to the appropriate routine is negligible. 

1.4. 3.1. A MIX simulator. When the language presented to an interpretive 
routine is the machine language of another computer, the interpreter is often 
called a simulator (or sometimes an emulator). 

In the author s opinion, entirely too much programmers’ time has been 
spent in wiiting such simulators and entirely too much computer time has been 
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wasted in using them. The motivation for simulators is simple: A computer 
installation buys a new machine and still wants to run programs written for 
the old machine (rather than rewriting the programs). However, this usually 
costs more and gives poorer results than if a special task force of programmers 
were given temporary employment to do the reprogramming. For example, the 
author once participated in such a reprogramming project, and a serious error 
was discovered in the original program, which had been in use for several years; 
the new program worked at five times the speed of the old, besides giving the 
right answers for a change! (Not all simulators are bad; for example, it is usually 
advantageous for a computer manufacturer to simulate a new machine before it 
has been built, so that software for the new machine may be developed as soon as 
possible. But that is a very specialized application.) An extreme example of the 
inefficient use of computer simulators is the true story of machine A simulating 
machine B running a program that simulates machine C\ This is the way to 
make a large, expensive computer give poorer results than its cheaper cousin. 

In view of all this, why should such a simulator rear its ugly head in this 
book? There are two reasons: 

a) The simulator we will describe below is a good example of a typical interpre- 
tive routine; the basic techniques employed in interpreters are illustrated here. 
It also illustrates the use of subroutines in a moderately long program. 

b) We will describe a simulator of the MIX computer, written in (of all things) 
the MIX language. This will facilitate the writing of MIX simulators for most 
computers, which are similar; the coding of our program intentionally avoids 
making heavy use of MlX-oriented features. A MIX simulator will be of advantage 
as a teaching aid in conjunction with this book and possibly others. 

Computer simulators as described in this section should be distinguished 
from discrete system simulators. Discrete system simulators are important pro- 
grams that will be discussed in Section 2.2.5. 

Now let’s turn to the task of writing a MIX simulator. The input to our 
program will be a sequence of MIX instructions and data, stored in locations 
0000-3499. We want to mimic the precise behavior of Mix’s hardware, pretending 
that MIX itself is interpreting those instructions; thus, we want to implement 
the specifications that were laid down in Section 1.3.1. Our program will, for 
example, maintain a variable called AREG that will hold the magnitude of the 
simulated A-register; another variable, SIGNA, will hold the corresponding sign. 
A variable called CLOCK will record how many MIX units of simulated time have 
elapsed during the simulated program execution. 

The numbering of Mix’s instructions LDA, LD1, ..., LDX and other similar 
commands suggests that we keep the simulated contents of these registers in 
consecutive locations, as follows: 

AREG, I1REG, I2REG, I3REG, I4REG, I5REG, I6REG, XREG, JREG, ZERO. 

Here ZERO is a “register” filled with zeros at all times. The positions of JREG and 
ZERO are suggested by the op-code numbers of the instructions STJ and STZ. 
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In keeping with our philosophy of writing the simulator as though it were not 
really done with MIX hardware, we will treat the signs as independent parts of 
a register. For example, many computers cannot represent the number “minus 
zero”, while MIX definitely can; therefore we will always treat signs specially 
in this program. The locations AREG, I1REG, . . . , ZERO will always contain the 
absolute values of the corresponding register contents; another set of locations in 
our program, called SIGNA, SIGN1, . . . , SIGNZ will contain +1 or -1, depending 
on whether the sign of the corresponding register is plus or minus. 

An interpretive routine generally has a central control section that is called 
into action between interpreted instructions. In our case, the program transfers 
to location CYCLE at the end of each simulated instruction. 

The control routine does the things common to all instructions, unpacks the 
instruction into its various parts, and puts the parts into convenient places for 
later use. The program below sets 

rI6 = location of the next instruction; 

rI5 = M (address of the present instruction, plus indexing); 

rI4 = operation code of the present instruction; 

rI3 = F-field of the present instruction; 

INST = the present instruction. 


Program M . 



001 

* MIX SIMULATOR 


002 

ORIG 

3500 

Simulated memory is in locations 0000 up. 

003 

BEGIN STZ 

TIME(0 : 2) 


004 

STZ 

0VT0G 

0VT0G is the simulated overflow toggle. 

005 

STZ 

COMPI 

COMPI, ±1 or 0, is comparison indicator. 

006 

ENT6 

0 

Take first instruction from location zero. 

007 

CYCLE LDA 

CLOCK 

Beginning of control routine: 

008 

TIME INCA 

0 

This address is set to the execution time 

009 

STA 

CLOCK 

of the previous instruction (see line 033) 

010 

LDA 

0,6 

rA t— instruction to simulate. 

on 

STA 

INST 


012 

INC6 

1 

Advance the location counter. 

013 

LDX 

INST(1 : 2) 

Get absolute value of the address. 

014 

SLAX 

5 

Attach sign to the address. 

015 

STA 

M 


016 

LD2 

INST(3 : 3) 

Examine the index field. 

017 

J2Z 

IF 

Is it zero? 

018 

DEC2 

6 


019 

J2P 

INDEXERROR 

Illegal index specified? 

020 

LDA 

SIGN6 , 2 

Get sign of the index register. 

021 

LDX 

I6REG , 2 

Get magnitude of the index register. 

022 

SLAX 

5 

Attach the sign. 

023 

ADD 

M 

Do signed addition for indexing. 

024 

CMP A 

ZERO (1:3) 

Is the result too large? 

025 

JNE 

ADDRERROR 

If so, simulate an error. 

026 

STA 

M 

Otherwise the address has been found. 
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027 1H 

LD3 

INST (4 : 4) 

028 

LD5 

M 

029 

LD4 

INST (5 : 5) 

030 

DEC4 

63 

031 

J4P 

0PERR0R 

032 

LDA 

OPTABLE, 4 (4: 4) 

033 

STA 

TIME(0:2) 

034 

LD2 

OPTABLE, 4 (0:2) 

035 

JNOV 

0,2 

036 

JMP 

0,2 
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rI3 <— F-field. 
rI5 4— M. 
rI4 4— C-field. 

Is the op code > 64? 

Get execution time from the table. 

Get address of the proper routine. 

Jump to operator. 

(Protect against overflows.) | 


The reader’s attention is called particularly to lines 034 -036: A “switching 
table” of the 64 operators is part of the simulator, allowing it to jump rapidly to 
the correct routine for the current instruction. This is an important time-saving 
technique (see exercise 1.3. 2-9). 

The 64-word switching table, called OPTABLE, gives also the execution time 
for the various operators; the following lines indicate the contents of that table: 


037 

NOP 

CYCLE (1) 

Operation code table; 

038 

ADD 

ADD (2) 

typical entry is 

039 

SUB 

SUB (2) 

“OP routine(time)” 

040 

MUL 

MUL (10) 


041 

DIV 

DIV(12) 


042 

HLT 

SPEC(l) 


043 

SLA 

SHIFT(2) 


044 

MOVE 

M0VE(1) 


045 

LDA 

L0AD(2) 


046 

LD1 

LOAD, 1(2) 



051 LD6 

052 LDX 

053 LDAN 

054 LD1N 

060 LDXN LOADN (2) 

061 STA STORE (2) 

069 STJ 

070 STZ 

Oil JBUS 

072 IOC 

073 IN 

074 OUT 

075 JRED 

076 JMP 

077 JAP 

084 JXP REGJUMP(l) 

085 INCA ADDROP(l) 


ST0RE(2) 

ST0REC2) 

JBUS(l) 

I0C(1) 

IN(1) 

OUT ( 1 ) 

JRED (1) 
JUMP ( 1 ) 

REG JUMP (1) 


LOAD, 1(2) 
LOAD (2) 
LOADN (2) 
LOADN, 1(2) 
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086 

INC1 

ADDROP ,1(1) 

092 

INCX 

ADDROP(l) 

093 

CMPA 

COMPARE (2) 

100 

OPTABLE CMPX 

COMPARE (2) 


(The entries for operators LDi, LDiN, and INC* have an additional “,1” to set the 
(3:3) field nonzero; this is used below in lines 289-290 to indicate the fact that 
the size of the quantity within the corresponding index register must be checked 
after simulating these operations.) 

The next part of our simulator program merely lists the locations used to 
contain the contents of the simulated registers: 


101 

AREG 

CON 

0 

Magnitude of A-register 

102 

11 REG 

CON 

0 

Magnitude of index registers 

107 

I6REG 

CON 

0 


108 

XREG 

CON 

0 

Magnitude of X-register 

109 

JREG 

CON 

0 

Magnitude of J-register 

110 

ZERO 

CON 

0 

Constant zero, for “STZ” 

111 

SIGNA 

CON 

1 

Sign of A-register 

112 

SIGN1 

CON 

1 

Sign of index registers 

117 

SIGN6 

CON 

1 


118 

SIGNX 

CON 

1 

Sign of X-register 

119 

SIGNJ 

CON 

1 

Sign of J-register 

120 

SIGNZ 

CON 

1 

Sign stored by “STZ” 

121 

INST 

CON 

0 

Instruction being simulated 

122 

COMP I 

CON 

0 

Comparison indicator 

123 

0VT0G 

CON 

0 

Overflow toggle 

124 

CLOCK 

CON 

0 

Simulated execution time | 


Now we will consider three subroutines used by the simulator. First comes 
the MEMORY subroutine: 

Calling sequence: JMP MEMORY. 

Entry conditions: rI5 = valid memory address (otherwise the subroutine will 
jump to MEMERROR). 

Exit conditions: rX = sign of word in memory location rI5; rA = magnitude of 
word in memory location rI5. 

125 * SUBROUTINES 


126 MEMORY 

STJ 

9F 

Memory fetch subroutine: 

127 

J5N 

MEMERROR 


128 

CMP 5 

=BEGIN= 

The simulated memory is in 

129 

JGE 

MEMERROR 

locations 0000 to BEGIN 

130 

LDX 

0,5 


131 

ENTA 

1 


132 

SRAX 

5 

rX «— sign of word. 


1.4.3. 1 


A MIX SIMULATOR 207 


133 LDA 0,5(1: 5) rA e- magnitude of word. 

134 9H JMP * Exit. | 

The FCHECK subroutine processes a partial field specification, making sure 
that it has the form 8L + R with L < R < 5. 

Calling sequence: JMP FCHECK . 

Entry conditions: rI3 = valid field specification (otherwise the subroutine will 
jump to FERROR). 

Exit conditions: rA = rll = L, rX = R. 


135 

FCHECK 

STJ 

9F 

Field check subroutine: 

136 


ENTA 

0 


137 


ENTX 

0,3 

rAX c— field specification 

138 


DIV 

=8= 

rA -f— L, rX <— R. 

139 


CMPX 

=5= 

Is R > 5? 

140 


JG 

FERROR 


141 


STX 

R 


142 


STA 

L 


143 


LD1 

L 

rll <r- L. 

144 


CMP A 

R 


145 

9H 

JLE 

* 

Exit unless L > R. 

146 


JMP 

FERROR 

1 


The last subroutine, GETV, finds the quantity V (namely, the appropriate 
field of location M) used in various MIX operators, as defined in Section 1.3.1. 

Calling sequence: JMP GETV. 

Entry conditions: rI5 = valid memory address; rI3 = valid field. (If invalid, an 
error will be detected as above.) 

Exit conditions: rA = magnitude of V; rX = sign of V; rll = L; rI2 = — R. 
Second entrance: JMP GETAV, used only in comparison operators to extract a 
field from a register. 


147 

GETAV 

STJ 

9F 

Special entrance, see line 300. 

148 


JMP 

IF 


149 

GETV 

STJ 

9F 

Subroutine to find V: 

150 


JMP 

FCHECK 

Process the field and set rll <— L. 

151 


JMP 

MEMORY 

rA memory magnitude, rX <— sign. 

152 

1H 

J1Z 

2F 

Is the sign included in the field? 

153 


ENTX 

1 

If not, set the sign positive. 

154 


SLA 

-1,1 

Zero out all bytes to the left 

155 


SRA 

"1,1 

of the field. 

156 

2H 

LD2N 

R 

Shift right into the 

157 


SRA 

5,2 

proper position. 

158 

9H 

JMP 

* 

Exit. | 


Now we come to the routines for each individual operator. These routines 
are given here for completeness, but the reader should study only a few of them 
unless there’s a compelling reason to look closer; the SUB and JUMP operators are 
recommended as typical examples for study. Notice the way in which routines 
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for similar operations can be neatly combined, and notice how the JUMP routine 
uses another switching table to govern the type of jump. 


159 

* INDIVIDUAL 

OPERATORS 


160 

ADD 

JMP 

GETV 

Get the value of V in rA and rX. 

161 


ENT1 

0 

rll ■<— index of simulated rA. 

162 


JMP 

INC 

Go to the “increase” routine. 

163 

SUB 

JMP 

GETV 

Get the value of V in rA and rX. 

164 


ENT1 

0 

rll «— index of simulated rA. 

165 


JMP 

DEC 

Go to the “decrease” routine. 

166 

* 




167 

MUL 

JMP 

GETV 

Get the value of V in rA and rX. 

168 


CMPX 

SIGN A 

Are signs the same? 

169 


ENTX 

1 


170 


JE 

*+2 

Set rX to the result sign. 

171 


ENNX 

1 

172 


STX 

SIGNA 

Put it in both simulated registers 

173 


STX 

SIGNX 


174 


MUL 

AREG 

Multiply the operands. 

175 


JMP 

STOREAX 

Store the magnitudes. 

176 

* 




177 

DIV 

LDA 

SIGNA 

Set the sign of the remainder. 

178 


STA 

SIGNX 


179 


JMP 

GETV 

Get the value of V in rA and rX. 

180 


CMPX 

SIGNA 

Are signs the same? 

181 


ENTX 

1 


182 


JE 

*+2 

Set rX to the result sign. 

183 


ENNX 

1 

184 


STX 

SIGNA 

Put it in the simulated rA. 

185 


STA 

TEMP 


186 


LDA 

AREG 

Divide the operands. 

187 


LDX 

XREG 


188 


DIV 

TEMP 


189 

STOREAX 

STA 

AREG 

Store the magnitudes. 

190 


STX 

XREG 


191 

GVCHECK 

JNOV 

CYCLE 

Did overflow just occur? 

192 


ENTX 

1 

If so, set the simulated 

193 


STX 

OVTOG 

overflow toggle on. 

194 


JMP 

CYCLE 

Return to control routine. 

195 

* 




196 

LOADN 

JMP 

GETV 

Get the value of V in rA and rX. 

197 


ENT1 

47,4 

rll «— C — 16; indicates register. 

198 

L0ADN1 

STX 

TEMP 

Negate the sign. 

199 


LDXN 

TEMP 


200 


JMP 

L0AD1 

Change LOADN to LOAD. 

201 

LOAD 

JMP 

GETV 

Get the value of V in rA and rX. 

202 


ENT1 

55,4 

rll <— C — 8, indicates register. 

203 

L0AD1 

STA 

AREG , 1 

Store the magnitude. 

204 


STX 

SIGNA, 1 

Store the sign. 
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205 


JMP 

SIZECHK 

206 

* 



207 

STORE 

JMP 

FCHECK 

208 


JMP 

MEMORY 

209 


J1P 

IF 

210 


ENT1 

1 

211 


LDX 

SIGNA+39,4 

212 

1H 

LD2N 

R 

213 


SRAX 

5,2 

214 


LDA 

AREG+39 , 4 

215 


SLAX 

5,2 

216 


ENN2 

0,1 

21 7 


SRAX 

6,2 

218 


LDA 

0,5 

219 


SRA 

6,2 

220 


SRAX 

"1,1 

221 


STX 

0,5 

222 


JMP 

CYCLE 

223 

* 



224 

JUMP 

DEC3 

9 

225 


J3P 

FERROR 

226 


LDA 

COMPI 

227 


JMP 

JTABLE , 3 

228 

JMP 

ST6 

JREG 

229 


JMP 

JSJ 

230 


JMP 

JOV 

231 


JMP 

JNOV 

232 


JMP 

LS 

233 


JMP 

EQ 

234 


JMP 

GR 

235 


JMP 

GE 

236 


JMP 

NE 

237 

JTABLE 

JMP 

LE 

238 

JOV 

LDX 

0VT0G 

239 


JMP 

*+3 

240 

JNOV 

LDX 

0VT0G 

241 


DECX 

1 

242 


STZ 

0VT0G 

243 


JXNZ 

JMP 

244 


JMP 

CYCLE 

245 

LE 

JAZ 

JMP 

246 

LS 

JAN 

JMP 

247 


JMP 

CYCLE 

248 

NE 

JAN 

JMP 

249 

GR 

JAP 

JMP 

250 


JMP 

CYCLE 

251 

GE 

JAP 

JMP 

252 

EQ 

JAZ 

JMP 

253 


JMP 

CYCLE 


Check if the magnitude is too large 
rll <- L. 

Get contents of memory location. 

Is the sign included in the field? 

If so, change L to 1 

and “store” the register’s sign. 
rI2 f- -R. 

Save the area to the field’s right. 
Insert register in the field. 

rI2 < L. 

Restore the area to the field’s left. 

Attach the sign. 

Store in memory. 

Return to control routine. 

Jump operators: 

Is F too large? 

rA <— comparison indicator. 

Jump to appropriate routine. 

Set the simulated J-register. 


End of the jump table 
Check whether to jump on 
overflow. 

Get complement of overflow toggle. 
Shut off overflow toggle. 

Jump. 

Don’t jump. 

Jump if rA zero or negative. 

Jump if rA negative. 

Don’t jump. 

Jump if rA negative or positive. 
Jump if rA positive. 

Don’t jump. 

Jump if rA positive or zero. 

Jump if rA zero. 

Don’t jump. 
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254 

JSJ 

JMP 

MEMORY 

255 


ENT6 

0,5 

256 


JMP 

CYCLE 

257 

* 



258 

REG JUMP 

LDA 

AREG+23,4 

259 


JAZ 

*+2 

260 


LDA 

SIGNA+23,4 

261 


DEC3 

5 

262 


J3NP 

JTABLE.3 

263 


JMP 

FERROR 

264 

* 



265 

ADDROP 

DEC3 

3 

266 


J3P 

FERROR 

267 


ENTX 

0,5 

268 


JXNZ 

*+2 

269 


LDX 

INST 

270 


ENTA 

1 

271 


SRAX 

5 

272 


LDA 

M(1 : 5) 

273 


ENT1 

15,4 

274 


JMP 

IF, 3 

275 


JMP 

INC 

276 


JMP 

DEC 

277 


JMP 

L0AD1 

278 

1H 

JMP 

L0ADN1 

279 

DEC 

STX 

TEMP 

280 


LDXN 

TEMP 

281 

INC 

CMPX 

SIGNA.l 

282 


JE 

IF 

283 


SUB 

AREG.l 

284 


JANP 

2F 

285 


STX 

SIGNA.l 

286 


JMP 

2F 

287 

1H 

ADD 

AREG , 1 

288 

2H 

STA 

AREG, 1(1:5) 

289 

SIZECHK 

LD1 

0PTABLE,4(3:3) 

290 


J1Z 

0VCHECK 

291 


CMPA 

ZERO (1:3) 

292 


JE 

CYCLE 

293 


JMP 

SIZEERR0R 

294 

* 



295 

COMPARE 

JMP 

GETV 

296 


SRAX 

5 

297 


STX 

V 

298 


LDA 

XREG , 4 

299 


LDX 

SIGNX.4 

300 


JMP 

GETAV 

301 


SRAX 

5 

302 


CMPX 

V 
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Check for valid memory address. 

Simulate a jump. 

Return to main control routine. 

Register jumps: 

Is register zero? 

If not, put sign into rA. 

Change to a conditional JMP, unless 
the F-specification is too large. 

Address transfer operators: 

Is F too large? 

Find the sign of M. 

rX «— sign of M. 
rA <— magnitude of M. 
rll indicates the register. 

Four- way jump. 

Increase. 

Decrease. 

Enter. 

Enter negative. 

Reverse the sign. 

Reduce DEC to INC. 

Addition routine: 

Are signs the same? 

No; subtract magnitudes. 

Sign change needed? 

Change the register’s sign. 

Add magnitudes. 

Store magnitude of the result. 

Have we just loaded an 
index register? 

If so, make sure that the result 
fits in two bytes. 

Get the value of V in rA and rX. 

Attach the sign. 

Get field F of the appropriate register. 

Attach the sign. 

Compare (note that —0 = +0). 
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303 

STZ 

COMPI 

Set comparison indicator to 

304 

JE 

CYCLE 

either zero, plus one, 

305 

ENTA 

1 

or minus one. 

306 

JG 

*+2 


307 

ENNA 

1 


308 

STA 

COMPI 


309 

JMP 

CYCLE 

Return to control routine. 

310 * 

311 

END 

BEGIN | 



The code above adheres to a subtle rule that was stated in Section 1.3.1: 
The instruction “ENTA -0” loads minus zero into register A, as does “ENTA -5 , 1” 
when index register 1 contains +5. In general, when M is zero, ENTA loads the 
sign of the instruction and ENNA loads the opposite sign. The need to specify this 
condition was overlooked when the author prepared his first draft of Section 1.3.1; 
such questions usually come to light only when a computer program is being 
written to follow the rules. 

In spite of its length, the program above is incomplete in several respects: 

a) It does not recognize floating point operations. 

b) The coding for operation codes 5, 6, and 7 has been left as an exercise. 

c) The coding for input-output operators has been left as an exercise. 

d) No provision has been made for loading simulated programs (see exercise 4). 

e) The error routines 

INDEXERROR, ADDRERROR, 0PERR0R, MEMERROR, FERROR, SIZEERROR 

have not been included; they handle error conditions that are detected in 
the simulated program. 

f) There is no provision for diagnostic facilities. (A useful simulator should, 
for example, make it possible to print out the register contents as a program 
is being executed.) 

EXERCISES 

1. [ 14 ] Study all the uses of the FCHECK subroutine in the simulator program. Can 
you suggest a better way to organize the code? (See step 3 in the discussion at the end 
of Section 1.4.1.) 

2. [ 20 ] Write the SHIFT routine, which is missing from the program in the text 
(operation code 6). 

► 3. [ 22 ] Write the MOVE routine, which is missing from the program in the text 
(operation code 7). 

4. [ 14 ] Change the program in the text so that it begins as though Mix’s “GO button” 
had been pushed (see exercise 1.3.1-26). 

► 5. [ 24 ] Determine the time required to simulate the LDA and ENTA operators, com- 
pared with the actual time for MIX to execute these operators directly. 

6. [ 28 ] Write programs for the input-output operators JBUS, IOC, IN, OUT, and JRED, 
which are missing from the program in the text, allowing only units 16 and 18. Assume 
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that the operations u read-card” and “skip-to-new-page” take T = lOOOOu, while “print- 
line” takes T = 7500«. [Note: Experience shows that the JBUS instruction should be 
simulated by treating “JBUS *” as a special case; otherwise the simulator seems to 
stop!] 

► 7. [32] Modify the solutions of the previous exercise in such a way that execution 

of IN or OUT does not cause I/O transmission immediately; the transmission should 
take place after approximately half of the time required by the simulated devices has 
elapsed. (1 his will prevent a frequent student error, in which IN and OUT are used 
improperly.) 

8. [20] True or false: Whenever line 010 of the simulator program is executed we 
have 0 < rI6 < BEGIN. 


*1.4. 3. 2. Trace routines. When a machine is being simulated on itself (as 
MIX was simulated on MIX in the previous section) we have the special case of 
a simulator called a trace or monitor routine. Such programs are occasionally 
used to help in debugging, since they print out a step-by-step account of how 
the simulated program behaves. 

The program in the preceding section was written as though another com- 
puter were simulating MIX. A quite different approach is used for trace programs; 
we generally let registers represent themselves and let the operators perform 
themselves. In fact, we usually contrive to let the machine execute most of 
the instructions by itself. The chief exception is a jump or conditional jump 
instruction, which must not be executed without modification, since the trace 
program must remain in control. Each machine also has idiosyncratic features 
that make tracing more of a challenge; in Mix’s case, the J-register presents the 
most interesting problem. 

The trace routine given below is initiated when the main program jumps to 
location ENTER, with register J set to the address for starting to trace and reg- 
ister X set to the address where tracing should stop. The program is interesting 
and merits careful study. 


01 

* TRACE ROUTINE 


02 

ENTER 

STX 

TEST(0 : 2) 

Set the exit location. 

03 


STX 

LEAVEX(0 : 2) 


04 


STA 

AREG 

Save the contents of rA. 

05 


STJ 

JREG 

Save the contents of rj. 

06 


LDA 

JREG(0 : 2) 

Get the start location for tracing. 

07 

CYCLE 

STA 

PREG(0 : 2) 

Store the location of the next instruction. 

08 

TEST 

DECA 

* 

Is it the exit location? 

09 


JAZ 

LEAVE 


10 

PREG 

LDA 

* 

Get the next instruction. 

11 


STA 

INST 

Copy it. 

12 


SRA 

2 


13 


STA 

INST1 (0:3) 

Store the address and index parts. 

u 


LDA 

INST(5 : 5) 

Get the operation code, C. 

15 


DECA 

38 


16 


JANN 

IF 

Is C > 38 ( JRED)? 
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17 


INCA 

6 


18 


JANZ 

2F 

Is C / 32 (STJ)? 

19 


LDA 

INST(0 : 4) 


20 


STA 

*+2(0:4) 

Change STJ to STA. 

21 

JREG 

ENTA 

* 

rA <— simulated rJ contents. 

22 


STA 

* 


23 


JMP 

INCP 


24 

2H 

DECA 

2 


25 


JANZ 

2F 

Is C / 34 ( JBUS)? 

26 


JMP 

3F 


27 

1H 

DECA 

9 

Test for jump instructions. 

28 


JAP 

2F 

Is C > 47 ( JXNP)? 

29 

3H 

LDA 

8F (0 : 3) 

We detected a jump instruction; 

30 


STA 

INST(0:3) 

change its address to “JUMP” . 

31 

2H 

LDA 

AREG 

Restore register A. 

32 

* 



All registers except ,1 now have proper 

33 

* 



values with respect to the external program. 

34 

INST 

NOP 

* 

The instruction is executed. 

35 


STA 

AREG 

Store register A again. 

36 

INCP 

LDA 

PREG(0 : 2) 

Move to the next instruction. 

37 


INCA 

1 


38 


JMP 

CYCLE 


39 

8H 

JSJ 

JUMP 

Constant for lines 29 and 40 

40 

JUMP 

LDA 

8B (4 : 5) 

A jump has occurred. 

41 


SUB 

INST (4 : 5) 

Was it JSJ? 

42 


JAZ 

*+4 


43 


LDA 

PREG(0:2) 

If not, update the simulated 

44 


INCA 

1 

J-register. 

45 


STA 

JREG (0:2) 


46 

INST1 

ENTA 

* 


47 


JMP 

CYCLE 

Move to the address of the jump. 

48 

LEAVE 

LDA 

AREG 

Restore register A. 

49 

LEA VEX 

JMP 

* 

Stop tracing. 

50 

AREG 

CON 

0 

Simulated rA contents | 


The following things should be noted about trace routines in general and 
this one in particular. 

1) We have presented only the most interesting part of a trace program, 
the part that retains control while executing another program. For a trace to 
be useful, there must also be a routine for writing out the contents of registers, 
and this has not been included. Such a routine distracts from the more subtle 
features of a trace program, although it certainly is important; the necessary 
modifications are left as an exercise (see exercise 2). 

2) Space is generally more important than time; that is, the program should 
be written to be as short as possible. Then the trace routine will be able to coexist 
with extremely large programs. The running time is consumed by output anyway. 

3) Care was taken to avoid destroying the contents of most registers; in fact, 
the program uses only Mix’s A-register. Neither the comparison indicator nor 
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the overflow toggle are affected by the trace routine. (The less we use, the less 
we need to restore.) 

4) When a jump to location JUMP occurs, it is not necessary to “STA AREG’% 
since rA cannot have changed. 

5) After leaving the trace routine, the J-register is not reset properly. Exer- 
cise 1 shows how to remedy this. 

6) The program being traced is subject to only three restrictions: 

a) It must not store anything into the locations used by the trace program. 

b) It must not use the output device on which tracing information is being 
recorded (for example, JBUS would give an improper indication). 

c) It will run at a slower speed while being traced. 

EXERCISES 

1. [22] Modify the trace routine of the text so that it restores register J when leaving. 
(You may assume that register J is not zero.) 

2. [26] Modify the trace routine of the text so that before executing each program 
step it writes the following information on tape unit 0. 

Word 1, (0:2) field: location. 

Word 1, (4:5) field: register J (before execution). 

Word 1, (3:3) field: 2 if comparison is greater, 1 if equal, 0 if less; plus 8 if overflow 
is not on before execution. 

Word 2: instruction. 

Word 3: register A (before execution). 

Words 4-9: registers 11-16 (before execution). 

Word 10: register X (before execution). 

Words I T 100 of each 100- word tape block should contain nine more ten- word groups, 
in the same format. 

3. [10] The previous exercise suggests having the trace program write its output 
onto tape. Discuss why this would be preferable to printing directly. 

► 4. [25] What would happen if the trace routine were tracing itself ? Specifically, 
consider the behavior if the two instructions ENTX LEAVEX; JMP *+l were placed just 
before ENTER. 

5. [28] In a manner similar to that used to solve the previous exercise, consider 
the situation in which two copies of the trace routine are placed in different places in 
memory, and each is set up to trace the other. What would happen? 

► 6. [ 40 ] Write a trace routine that is capable of tracing itself, in the sense of exercise 4: 
It should print out the steps of its own program at slower speed, and that program will 
be tracing itself at still slower speed, ad infinitum, until memory capacity is exceeded. 

► 7. [25] Discuss how to write an efficient jump trace routine, which emits much less 
output than a normal trace. Instead of displaying the register contents, a jump trace 
simply records the jumps that occur. It outputs a sequence of pairs (air , 2 / 1 ), (x 2 ,y 2 ), 

■ ■ ■ , meaning that the program jumped from location xi to yi, then (after performing 
the instructions in locations j/i, 2/1 + 1, . . . , x 2 ) it jumped from x 2 to y 2 , etc. [From 
this information it is possible for a subsequent routine to reconstruct the flow of the 
program and to deduce how frequently each instruction was performed.] 
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1.4.4. Input and Output 

Perhaps the most outstanding differences between one computer and the next are 
the facilities available for doing input and output, and the computer instructions 
that govern those peripheral devices. We cannot hope to discuss in a single 
book all of the problems and techniques that arise in this area, so we will 
confine ourselves to a study of typical input-output methods that apply to most 
computers. The input-output operators of MIX represent a compromise between 
the widely varying facilities available in actual machines; to give an example of 
how to think about input-output, let us discuss in this section the problem of 
getting the best MIX input-output. 

/^\ Once again the reader is asked to be indulgent about the anachronistic 
jl MIX computer with its punched cards, etc. Although such old-fashioned 
devices are now quite obsolete, they still can teach important lessons. The MMIX 
computer, when it comes, will of course teach those lessons even better. 

Many computer users feel that input and output are not actually part of 
“real” programming; input and output are considered to be tedious tasks that 
people must perform only because they need to get information in and out of 
a machine. For this reason, the input and output facilities of a computer are 
usually not learned until after all other features have been examined, and it 
frequently happens that only a small fraction of the programmers of a particular 
machine ever know much about the details of input and output. This attitude 
is somewhat natural, because the input-output facilities of machines have never 
been especially pretty. However, the situation cannot be expected to improve 
until more people give serious thought to the subject. We shall see in this section 
and elsewhere (for example, in Section 5.4.6) that some very interesting issues 
arise in connection with input-output, and some pleasant algorithms do exist. 

A brief digression about terminology is perhaps appropriate here. Although 
dictionaries of English formerly listed the words “input” and “output” only as 
nouns (“What kind of input are we getting?”), it is now customary to use them 
grammatically as adjectives (“Don’t drop the input tape.”) and as transitive 
verbs (“Why did the program output this garbage?”). The combined term 
“input-output” is most frequently referred to by the abbreviation “I/O”. In- 
putting is often called reading, and outputting is, similarly, called writing. The 
stuff that is input or output is generally known as “data” — this word is, strictly 
speaking, a plural form of the word “datum,” but it is used collectively as if it 
were singular (“The data has not been read.”), just as the word “information” 
is both singular and plural. This completes today’s English lesson. 

Suppose now that we wish to read from magnetic tape. The IN operator 
of MIX, as defined in Section 1.3.1, merely initiates the input process, and the 
computer continues to execute further instructions while the input is taking 
place. Thus the instruction “IN 1000(5)” will begin to read 100 words from 
tape unit number 5 into memory cells 1000-1099, but the ensuing program must 
not refer to these memory cells until later. The program can assume that input 
is complete only after (a) another I/O operation (IN, OUT, or IOC) referring 
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to unit 5 has been initiated, or (b) a conditional jump instruction JBUS(5) or 
JRED(5) indicates that unit 5 is no longer “busy.” 

The simplest way to read a tape block into locations 1000-1099 and to have 
the information present is therefore the sequence of two instructions 


IN 1000(5); JBUS *(5). (i) 

We have used this rudimentary method in the program of Section 1.4.2 (see lines 
07-08 and 52-53). The method is generally wasteful of computer time, however, 
because a very large amount of potentially useful calculating time, say lOOOu or 
even lOOOOu, is consumed by repeated execution of the “JBUS” instruction. The 
program’s running speed can be as much as doubled if this additional time is 
utilized for calculation. (See exercises 4 and 5.) 

One way to avoid such a “busy wait” is to use two areas of memory for the 
input: We can read into one area while computing with the data in the other. 
For example, we could begin our program with the instruction 


IN 2000(5) Begin reading first block. ( 2 ) 

Subsequently, we may give the following five commands whenever a tape block 
is desired: 


ENT1 1000 
JBUS *(5) 

MOVE 2000(50) 
MOVE 2050(50) 
IN 2000(5) 


Prepare for MOVE operator. 
Wait until unit 5 is ready. 
(2000-2049) — t (1000-1049) 
(2050-2099) -> (1050-1099) 
Begin reading next block. 


(3) 


This has the same overall effect as (l), but it keeps the input tape busy while 
the program works on the data in locations 1000-1099. 

The last instruction of ( 3 ) begins to read a tape block into locations 2000- 
2099 before the preceding block has been examined. This is called “reading 
ahead” or anticipated input- it is done on faith that the block will eventually be 
needed. In fact, however, we might discover that no more input is really required, 
after we begin to examine the block in 1000—1099. For example, consider the 
analogous situation in the coroutine program of Section 1 . 4 . 2 , where the input 
was coming from punched cards instead of tape: A “.” appearing anywhere in 
the card meant that it was the final card of the deck. Such a situation would 
make anticipated input impossible, unless we could assume that either (a) a 
blank card or special trailer card of some other sort would follow the input deck, 
or (b) an identifying mark (e.g., “.”) would appear in, say, column 80 of the final 
card of the deck. Some means for terminating the input properly at the end of 
the program must always be provided whenever input has been anticipated. 

The technique of overlapping computation time and I/O time is known as 
buffering, while the rudimentary method ( 1 ) is called unbuffered input. The 
area of memory 2000-2099 used to hold the anticipated input in ( 3 ), as well as 
the area 1000-1099 to which the input was moved, is called a buffer. Webster’s 
New World Dictionary defines “buffer” as “any person or thing that serves to 
lessen shock,” and the term is appropriate because buffering tends to keep I/O 
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devices running smoothly. (Computer engineers often use the word “buffer” in 
another sense, to denote a part of the I/O device that stores information during 
the transmission. In this book, however, “buffer” will signify an area of mem,ory 
used by a programmer to hold I/O data.) 

The sequence (3) is not always superior to (1), although the exceptions are 
rare. Let us compare the execution times: Suppose T is the time required to 
input 100 words, and suppose C is the computation time that intervenes between 
input requests. Method (1) requires a time of essentially T + C per tape block, 
while method (3) takes essentially max(C, T) + 202-u. (The quantity 202u is the 
time required by the two MOVE instructions.) One way to look at this running 
time is to consider “critical path time” — in this case, the amount of time the 
I/O unit is idle between uses. Method (1) keeps the unit idle for C units of time, 
while method (3) keeps it idle for 202 units (assuming that C <T). 

The relatively slow MOVE commands of ( 3 ) are undesirable, particularly 
because they take up critical path time when the tape unit must be inactive. 
An almost obvious improvement of the method allows us to avoid these MOVE 
instructions: The outside program can be revised so that it refers alternately 
to locations 1000-1099 and 2000-2099. While we are reading into one buffer 
area, we can be computing with the information in the other; then we can begin 
reading into the second buffer while computing with the information in the first. 
This is the important technique known as buffer swapping. The location of the 
current buffer of interest will be kept in an index register (or, if no index registers 
are available, in a memory location). We have already seen an example of buffer 
swapping applied to output in Algorithm 1.3.2P (see steps P9-P11) and the 
accompanying program. 

As an example of buffer swapping on input, suppose that we have a computer 
application in which each tape block consists of 100 separate one-word items. The 
following program is a subroutine that gets the next word of input and begins 
to read in a new block if the current one is exhausted. 


01 

WORD IN 

STJ 

IF 

Store the exit location. 

02 


INC 6 

1 

Advance to the next word. 

03 

2H 

LDA 

0,6 

Is it the end of the 

04 


CMPA 

=SENTINEL= 

buffer? 

05 

1H 

JNE 

* 

If not, exit. 

06 


IN 

-100,6(U) 

Refill this buffer. 

07 


LD 6 

1,6 

Switch to the other 

08 


JMP 

2B 

buffer and return. 

09 

INBUF1 

0RIG 

*+100 

First buffer 

10 


CON 

SENTINEL 

Sentinel at end of buffer 

11 


CON 

*+l 

Address of other buffer 

12 

INBUF2 

0RIG 

*+100 

Second buffer 

13 


CON 

SENTINEL 

Sentinel at end of buffer 

H 


CON 

INBUF1 

Address of other buffer | 


In this routine, index register 6 is used to address the last word of input; we 
assume that the calling program does not affect this register. The symbol U 
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refers to a tape unit, and the symbol SENTINEL refers to a value that is known 
(from characteristics of the program) to be absent from all tape blocks. 

Several things about this subroutine should be noted: 

1) The sentinel constant appears as the 101st word of each buffer, and it 
makes a convenient test for the end of the buffer. In many applications, however, 
the sentinel technique will not be reliable, since any word may appear on tape. 
If we were doing card input, a similar method (with the 17th word of the buffer 
equal to a sentinel) could always be used without fear of failure; in that case, 
any negative word could serve as a sentinel, since MIX input from cards always 
gives nonnegative words. 

2) Each buffer contains the address of the other buffer (see lines 07, 11, 
and 14). This “linking together” facilitates the swapping process. 

3) No JBUS instruction was necessary, since the next input was initiated 
before any word of the previous block was accessed. If the quantities C and T 
refer as before to computation time and tape time, the execution time per tape 
block is now max (C, T); it is therefore possible to keep the tape going at full 
speed if C < T. (Note: MIX is an idealized computer in this regard, however, 
since no I/O errors must be treated by the program. On most machines some 
instructions to test the successful completion of the previous operation would be 
necessary just before the “IN” instruction here.) 

4) To make subroutine ( 4 ) work properly, it will be necessary to get things 
started out right when the program begins. Details are left to the reader (see 
exercise 6 ). 

5) The WORDIN subroutine makes the tape unit appear to have a block length 
of 1 rather than 100 as far as the rest of the program is concerned. The idea 
of having several program-oriented records filling a single actual tape block is 
called blocking of records. 

The techniques that we have illustrated for input apply, with minor changes, 
to output as well (see exercises 2 and 3 ). 



Fig. 23. A circle of buffers ( N = 6). 


Multiple buffers. Buffer swapping is just the special case N = 2 of a general 
method involving N buffers. In some applications it is desirable to have more 
than two buffers; for example, consider the following type of algorithm: 
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Step 1 . Read five blocks in rapid succession. 

Step 2. Perform a fairly long calculation based on this data. 

Step 3. Return to step 1. 

Here five or six buffers would be desirable, so that the next batch of five blocks 
could be read during step 2. This tendency for I/O activity to be “bunched” 
makes multiple buffering an improvement over buffer swapping. 

Suppose we have TV buffers for some input or output process using a single 
I/O device; we will imagine that the buffers are arranged in a circle, as in 
Fig. 23. The program external to the buffering process can be assumed to have 
the following general form with respect to the I/O unit of interest: 


ASSIGN 

RELEASE 

ASSIGN 

RELEASE 


in other words, we can assume that the program alternates between an action 
called “ASSIGN” and an action called “RELEASE”, separated by other computa- 
tions that do not affect the allocation of buffers. 

ASSIGN means that the program acquires the address of the next buffer area; 
this address is assigned as the value of some program variable. 

RELEASE means that the program is done with the current buffer area. 

Between ASSIGN and RELEASE the program is communicating with one of the 
buffers, called the current buffer area; between RELEASE and ASSIGN, the pro- 
gram makes no reference to any buffer area. 

Conceivably, ASSIGN could immediately follow RELEASE, and discussions of 
buffering have often been based on this assumption. However, if RELEASE is done 
as soon as possible, the buffering process has more freedom and will be more 
effective; by separating the two essentially different functions of ASSIGN and 
RELEASE we will find that the buffering technique remains easy to understand, 
and our discussion will be meaningful even if TV = 1. 

To be more explicit, let us consider the cases of input and output separately. 
For input, suppose we are dealing with a card reader. The action ASSIGN means 
that the program needs to see information from a new card; we would like to set 
an index register to the memory address at which the next card image is located. 
The action RELEASE occurs when the information in the current card image is no 
longer needed — it has somehow been digested by the program, perhaps copied 
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to another part of memory, etc. The current buffer area may therefore be filled 
with further anticipated input. 

For output, consider the case of a line printer. The action ASSIGN occurs 
when a free buffer area is needed, into which a line image is to be placed for 
printing. We wish to set an index register equal to the memory address of such 
an area. The action RELEASE occurs when this line image has been fully set up 
in the buffer area, in a form ready to be printed. 

Example: To print the contents of locations 0800-0823, we might write 

JMP ASSIGNP (Sets rI5 to buffer location) 

ENT1 0,5 

MOVE 800(24) Move 24 words into the output buffer (^) 

JMP RELEASEP 


where ASSIGNP and RELEASEP represent subroutines to do the two buffering 
functions for the line printer. 

In an optimal situation, from the standpoint of the computer, the ASSIGN 
operation will require virtually no execution time. This means, on input, that 
each card image will have been anticipated, so that the data is available when 
the program is ready for it; and on output, it means that there will always be 
a free place in memory to record the line image. In either case, no time will be 
spent waiting for the I/O device. 

To help describe the buffering algorithm, and to make it more colorful, we 
will say that buffer areas are either green, yellow, or red (shown as G. Y, and R 
in Fig. 24). 

Green means that the area is ready to be ASSIGNed; this means that it has 
been filled with anticipated information (in an input situation), or that it is a 
free area (in an output situation). 

Yellow means that the area has been ASSIGNed, not RELEASEd; this means 
that it is the current buffer, and the program is communicating with it. 

Red means that the area has been RELEASEd; thus it is a free area (in an 
input situation) or it has been filled with information (in an output situation). 

Figure 23 shows two “pointers” associated with the circle of buffers. These 
are, conceptually, index registers in the program. NEXTG and NEXTR point to 
the next green and “next red” buffer, respectively. A third pointer, CURRENT 
(shown in Fig. 24), indicates the yellow buffer when one is present. 


The algorithms below apply equally well to input or output, but for defi- 
niteness we will consider first the case of input from a card reader. Suppose that 
a program has reached the state shown in Fig. 23. This means that four card 
images have been anticipated by the buffering process, and they reside in the 
green buffers. At this moment, two things are happening simultaneously : (a) The 
program is computing, following a RELEASE operation; (b) a card is being read 
into the buffer indicated by NEXTR. This state of affairs will continue until the 
input cycle is completed (the unit will then go from “busy” to “ready”), or until 
the program does an ASSIGN operation. Suppose the latter occurs first; then the 
buffer indicated by NEXTG changes to yellow (it is assigned as the current buffer). 
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Fig. 24. Buffer transitions, (a) after ASSIGN, (b) after I/O complete, and (c) after 
RELEASE. 


NEXTG moves clockwise, and we arrive at the position shown in Fig. 24(a). If 
now the input is completed, another anticipated block is present; so the buffer 
changes from red to green, and NEXTR moves over as shown in Fig. 24(b). If the 
RELEASE operation follows next, we obtain Fig. 24(c). 

For an example concerning output, see Fig. 27 on page 226. That illustration 
shows the “colors” of buffer areas as a function of time, in a program that opens 
with four quick outputs, then produces four at a slow pace, and finally issues two 
in rapid succession as the program ends. Three buffers appear in that example. 

The pointers NEXTR and NEXTG proceed merrily around the circle, each at an 
independent rate of speed, moving clockwise. It is a race between the program 
(which turns buffers from green to red) and the I/O buffering process (which 
turns them from red to green). Two situations of conflict can occur: 

a) if NEXTG tries to pass NEXTR, the program has gotten ahead of the 1/ O device 
and it must wait until the device is ready. 

b) if NEXTR tries to pass NEXTG, the I/O device has gotten ahead of the program 
and we must shut it down until the next RELEASE is given. 

Both of these situations are depicted in Fig. 27. (See exercise 9.) 

Fortunately, in spite of the rather lengthy explanation just given of the ideas 
behind a circle of buffers, the actual algorithms for handling the situation are 
quite simple. In the following description, 

N = total number of buffers; ^ 

n = current number of red buffers. 

The variable n is used in the algorithm below to avoid interference between 
NEXTG and NEXTR. 

Algorithm A (ASSIGN). This algorithm includes the steps implied by ASSIGN 
within a computational program, as described above. 

Al. [Wait for n < IV.] If n = N , stall the program until n < N. (If n = N, 
no buffers are ready to be assigned; but Algorithm B below, which runs in 
parallel with this one, will eventually succeed in producing a green buffer.) 
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A2. [CURRENT NEXTG.] Set CURRENT NEXTG (thereby assigning the current 
buffer). 

A3. [Advance NEXTG.] Advance NEXTG to the next clockwise buffer. | 

Algorithm R (RELEASE). This algorithm includes the steps implied by RELEASE 
within a computational program, as described above. 

Rl. [Increase n .] Increase n by one. | 

Algorithm B (Buffer control). This algorithm performs the actual initiation 
of I/O operators in the machine; it is to be executed “simultaneously” with the 
main program, in the sense described below. 

Bl. [Compute.] Let the main program compute for a short period of time; step 
B2 will be executed after a certain time delay, at a time when the I/O device 
is ready for another operation. 

B2. [ n = 0?] If n = 0, go to Bl. (Thus, if no buffers are red, no I/O action can 
be performed.) 

B3. [Initiate I/O.] Initiate transmission between the buffer area designated by 
NEXTR and the I/O device. 

B4. [Compute.] Let the main program run for a period of time; then go to 
step B5 when the I/O operation is completed. 

B5. [Advance NEXTR.] Advance NEXTR to the next clockwise buffer. 

B6. [Decrease n.] Decrease n by one, and go to B2. | 

In these algorithms, we have two independent processes going on “simultane- 
ously,” the buffering control program and the computation program. These pro- 
cesses are, in fact, coroutines , which we will call CONTROL and COMPUTE. Coroutine 
CONTROL jumps to COMPUTE in steps Bl and B4; coroutine COMPUTE jumps to 
CONTROL by interspersing “jump ready” instructions at sporadic intervals in its 
program. 

Coding this algorithm for MIX is extremely simple. For convenience, assume 
that the buffers are linked so that the word preceding each one is the address of 
the next; for example, with N = 3 buffers we have CONTENTS (BUF1 - 1) = BUF2. 
CONTENTS (BUF2 - 1) = BUF3, and CONTENTS (BUF3 - 1) = BUF1. 

Program A (ASSIGN, a subroutine within the COMPUTE coroutine). rI4 = 
CURRENT; rI6 = n; calling sequence is JMP ASSIGN; on exit, rX contains NEXTG. 


ASSIGN 

STJ 

9F 

Subroutine linkage 

1H 

JRED 

CONTROL (U) 

Al. Wait for n < N. 


CMP6 

=N= 



JE 

IB 



LD4 

NEXTG 

A2. CURRENT <- NEXTG. 


LDX 

-1,4 

A3. Advance NEXTG. 


STX 

NEXTG 


9H 

JMP 

* 

Exit. | 
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ASSIGN RELEASE 



BUFFER CONTROL 



Fig. 25. Algorithms for multiple buffering. 

Program R (RELEASE, code used within the COMPUTE coroutine). rI6 = n. This 
short code is to be inserted wherever RELEASE is desired. 

INC6 1 Rl. Increase n. 

JRED CONTROL (U) Possible jump to CONTROL coroutine | 

Program B ( The CONTROL coroutine ). rI6 = n, rI5 = NEXTR. 


CONTI 

JMP 

COMPUTE 

Bl. Compute. 

1H 

J6Z 

*-l 

B2. n = 0? 


IN 

0 , 5 (U) 

B3. Initiate I/O. 


JMP 

COMPUTE 

B4. Compute. 


LD5 

-1,5 

B5. Advance NEXTR. 


DEC6 

JMP 

1 

IB | 

B6. Decrease n. 


Besides the code above, we also have the usual coroutine linkage 

CONTROL STJ COMPUTEX COMPUTE STJ CONTROLX 

CONTROLX JMP CONTI COMPUTEX JMP C0MP1 

and the instruction “JRED CONTROL (U)” should be placed within COMPUTE about 
once in every fifty instructions. 

Thus the programs for multiple buffering essentially amount to only seven 
instructions for CONTROL, eight for ASSIGN, and two for RELEASE. 

It is perhaps remarkable that exactly the same algorithm will work for both 
input and output. What is the difference- how does the control routine know 
whether to anticipate (for input) or to lag behind (for output.)? The answer lies 
in the initial conditions: For input we start out with n = N (all buffers red) and 
for output we start out with n = 0 (all buffers green). Once the routine has been 
started properly, it continues to behave as either an input process or an output 
process, respectively. The other initial condition is that NEXTR = NEXTG, both 
pointing at one of the buffers. 

At the conclusion of the program, it is necessary to stop the I/O process (if 
it is input) or to wait until it is completed (for output); details are left to the 
reader (see exercises 12 and 13). 
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It is important to ask wliat is the best value of IV to use. Certainly as IV 
gets larger, the speed of the program will not decrease, but it will not increase 
indefinitely either and so we come to a point of diminishing returns. Let us refer 
again to the quantities C and T, representing computation time between I/O 
operators and the 1/ O time itself. More precisely, let C be the amount of time 
between successive ASSIGNS, and let T be the amount of time needed to transmit 
one block. If C is always greater than T, then IV = 2 is adequate, for it is not 
hard to see that with two buffers we keep the computer busy at all times. If 
C is always less than T, then again IV = 2 is adequate, for we keep the I/O 
device busy at all times (except when the device has special timing constraints 
as in exercise 19). Larger values of N are therefore useful chiefly when C varies 
between small values and large values; the average number of consecutive small 
values, plus 1, may be right for IV, if the large values of C are significantly longer 
than T. (However, the advantage of buffering is virtually nullified if all input 
occurs at the beginning of the program and if all output occurs at the end.) 
If the time between ASSIGN and RELEASE is always quite small, the value of N 
may be decreased by 1 throughout the discussion above, with little effect on 
running time. 

This approach to buffering can be adapted in many ways, and we will 
mention a few of them briefly. So far we have assumed that only one I/O device 
was being used; in practice, of course, several devices will be in use at the 
same time. 

There are several ways to approach the subject of multiple units. In the 
simplest case, we can have a separate circle of buffers for each device. Each 
unit will have its own values of n, IV, NEXTR. NEXTG, and CURRENT, and its own 
CONTROL coroutine. This will give efficient buffering action simultaneously on 
every I/O device. 

It is also possible to “pool” buffer areas that are of the same size, so that two 
or more devices share buffers from a common list. This can be handled by using 
the linked memory techniques of Chapter 2, with all red input buffers linked 
together in one list and all green output buffers linked together in another. It 
becomes necessary to distinguish between input and output in this case, and 
to rewrite the algorithms without using n and N. The algorithm may get 
irrevocably stuck, if all buffers in the pool are filled with anticipated input; 
so a check should be made that there is always at least one buffer (preferably 
one for each device) that is not input-green; only if the COMPUTE routine is stalled 
at step A1 for some input device should we allow input into the final buffer of 
the pool from that device. 

Some machines have additional constraints on the use of input-output, units, 
so that it is impossible to be transmitting data from certain pairs of devices at 
the same time. (For example, several units might be attached to the computer by 
means of a single “channel.”) This constraint also affects our buffering routine; 
when we must choose which I/O unit to initiate next, how is the choice to be 
made? This is called the problem of “forecasting.” The best forecasting rule for 
the general case would seem to give preference to the unit whose buffer circle 
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Fig. 26. Input and output from the same circle. 

has the largest value of n/N, assuming that the number of buffers in the circles 
has been chosen wisely. 

Let’s conclude this discussion by taking note of a useful method for doing 
both input and output from the same buffer circle, under certain conditions. 
Figure 26 introduces a new kind of buffer, which has the color purple. In this 
situation, green buffers represent anticipated input; the program ASSIGNS and 
a green buffer becomes yellow, then upon RELEASE it turns red and represents 
a block to be output. The input and output processes follow around the circle 
independently as before, except that now we turn red buffers to purple after 
the output is done, and convert purple to green on input. It is necessary to 
ensure that none of the pointers NEXTG, NEXTR, NEXTP pass each other. At 
the instant shown in Fig. 26, the program is computing between ASSIGN and 
RELEASE, while accessing the yellow buffer; simultaneously, input is going into 
the buffer indicated by NEXTP; and output is coming from the buffer indicated 
by NEXTR. 

EXERCISES 

1. [05] (a) Would sequence (3) still be correct if the MOVE instructions were placed 
before the JBUS instruction instead of after it? (b) What if the MOVE instructions were 
placed after the IN command? 

2. [10] The instructions ‘‘OUT 1000(6); JBUS *(6)” may be used to output a tape 
block in an unbuffered fashion, just as the instructions (1) did this for input. Give a 
method analogous to (2) and (3) that buffers this output, by using MOVE instructions 
and an auxiliary buffer in locations 2000-2099. 

► 3. [22] Write a buffer-swapping output subroutine analogous to (4). The subroutine, 

called W0RD0UT, should store the word in rA as the next word of output, and if a buffer 
is full it should write 100 words onto tape unit V. Index register 5 should be used to 
refer to the current buffer position. Show the layout of buffer areas and explain what 
instructions (if any) are necessary at the beginning and end of the program to ensure 
that the first and last blocks are properly written. The final block should be filled out 
with zeros if necessary. 

4. [M20] Show that if a program refers to a single I/O device, we might be able to 
cut the running time in half by buffering the I/O, in favorable circumstances; but we 
can never decrease the running time by more than a factor of two, with respect to the 
time taken by unbuffered I/O. 
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Fig. 27. Output with three buffers (see exercise 9). 


► 5. [M21] Generalize the situation of the preceding exercise to the case when the 
program refers to n I/O devices instead of just one. 

6. [12] What instructions should be placed at the beginning of a program so that 
the WORDIN subroutine ( 4 ) gets off to the right start? (For example, index register 6 
must be set to something.) 

7. [22] Write a subroutine called WORDIN that is essentially like (4) except that it 
does not make use of a sentinel. 

8. [11] The text describes a hypothetical input scenario that leads from Fig. 23 
through parts (a), (b), and (c) of Fig. 24. Interpret the same scenario under the 
assumption that output to the line printer is being done, instead of input from cards. 
(For example, what things are happening at the time shown in Fig. 23?) 

► 9. [21] A program that leads to the buffer contents shown in Fig. 27 may be char- 
acterized by the following list of times: 

A ’ 1000 ’ R > 1000 > A , 1000, R, 1000, A, 1000, R, 1000, A, 1000, R, 1000. 

A ’ 7000 > R - 5000 ’ A 7000, R, 5000, A, 7000, R, 5000, A. 7000, R, 5000 

A , 1000, R, 1000, A, 2000, R, 1000. 

This list means “assign, compute for 1000-u, release, compute for lOOOu, assign 
compute for 2000u, release, compute for lOOOu.” The computation times given do not 
include any intervals during which the computer might have to wait for the output 
device to catch up (as at the fourth “assign” in Fig. 27). The output device operates 
at a speed of 7500u per block. 


85000 
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The following chart specifies the actions shown in Fig. 27 as time passes: 


Time 

Action 

Time 

Action 


0 

ASSIGN (BUF1) 

38500 

OUT BUF3 


1000 

RELEASE, OUT BUF1 

40000 

ASSIGN (BUF1) 


2000 

ASSIGN (BUF2) 

46000 

Output stops. 


3000 

RELEASE 

47000 

RELEASE, OUT 

BUF1 

4000 

ASSIGN (BUF3) 

52000 

ASSIGN (BUF2) 


5000 

RELEASE 

54500 

Output stops. 


6000 

ASSIGN (wait) 

59000 

RELEASE, OUT 

BUF2 

8500 

BUF1 assigned, OUT BUF2 

64000 

ASSIGN(BUF3) 


9500 

RELEASE 

65000 

RELEASE 


10500 

ASSIGN (wait) 

66000 

ASSIGN (BUF1) 


16000 

BUF2 assigned. OUT BUF3 

66500 

OUT BUF3 


23000 

RELEASE 

68000 

RELEASE 


23500 

OUT BUF1 

69000 

Computation stops. 

28000 

ASSIGN (BUF3) 

74000 

OUT BUF1 


31000 

35000 

OUT BUF2 

RELEASE 

81500 

Output stops. 



The total time required was therefore 81500m; the computer was idle from 6000-8500, 
10500-16000, and 69000-81500, or 20500m altogether; the output unit was idle from 
0-1000, 46000-47000, and 54500-59000, or 6500m. 

Make a time-action chart like the above for the same program, assuming that there 
are only two buffers. 

10. [21] Repeat exercise 9, except with four buffers. 

11. [21] Repeat exercise 9, except with just one buffer. 

12. [24] Suppose that the multiple buffering algorithm in the text is being used for 
card input, and suppose the input is to terminate as soon as a card with in column 
80 has been read. Show how the CONTROL coroutine (Algorithm B and Program B) 
should be changed so that input is shut off in this way. 

13. [20] What instructions should be included at the end of the COMPUTE coroutine 
in the text, if the buffering algorithms are being applied to output, to ensure that all 
information has been output from the buffers? 

► 14. [20] Suppose the computational program does not alternate between ASSIGN and 
RELEASE, but instead gives the sequence of actions . . . ASSIGN . . . ASSIGN . . . RELEASE . . . 
RELEASE. What effect does this have on the algorithms described in the text? Is it 
possibly useful? 

► 15. [22] Write a complete MIX program that copies 100 blocks from tape unit 0 to 
tape unit 1, using just three buffers. The program should be as fast as possible. 

16. [29] Formulate the “green-yellow-red-purple” algorithm, suggested by Fig. 26, 
in the manner of the algorithms for multiple buffering given in the text, using three 
coroutines (one to control the input device, one for the output device, and one for the 
computation). 

17. [40] Adapt the multiple-buffer algorithm to pooled buffers; build in methods that 
keep the process from slowing down, due to too much anticipated input. Try to make 
the algorithm as elegant as possible. Compare your method to nonpooling methods, 
applied to real-life problems. 
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► 18. [30] A proposed extension of MIX allows its computations to be interrupted, as 
explained below. Your task in this exercise is to modify Algorithms and Programs A, 
R, and B of the text so that they use these interrupt facilities instead of the “JRED'’ 
instructions. 

The new MIX features include an additional 3999 memory cells, locations -3999 
through -0001. The machine has two internal “states,” normal state and control state. 
In normal state, locations —3999 through —0001 are not admissible memory locations 
and the MIX computer behaves as usual. When an “interrupt” occurs, due to conditions 
explained later, locations -0009 through -0001 are set equal to the contents of Mix’s 
registers: rA in -0009; rll through rI6 in -0008 through -0003; rX in -0002; and rj, 
the overflow toggle, the comparison indicator, and the location of the next instruction 
are stored in —0001 as 



the machine enters control state, at a location depending on the type of interrupt. 

Location —0010 acts as a “clock’ : Every lOOOu of time, the number appearing 
in this location is decreased by one, and if the result is zero an interrupt to location 
—0011 occurs. 

The new MIX instruction “INT” (C = 5, F = 9) works as follows: (a) In normal state, 
an interrupt occurs to location -0012. (Thus a programmer may force an interrupt, 
to communicate with a control routine; the address of INT has no effect, although the 
control routine may use it for information to distinguish between types of interrupt.) 
(b) In control state, all MIX registers are loaded from locations —0009 to —0001, the 
computer goes into normal state, and it resumes execution. The execution time for INT 
is 2 u in each case. 

An IN, OUT, or IOC instruction given in control state will cause an interrupt to occur 
as soon as the I/O operation is completed. The interrupt goes to location -(0020+ 
unit number). 

No interrupts occur while in control state; any interrupt conditions are “saved” 
until after the next INT operation, and interrupt will occur after one instruction of the 
normal state program has been performed. 

► 19. [M28] Special considerations arise when input or output involves short blocks on 
a rotating device like a magnetic disk. Suppose a program works with n > 2 consecutive 
blocks of information in the following way: Block k begins to be input at time t k , where 
t\ = 0. It is assigned for processing at time u k > tk +T and released from its buffer at 
time v k = u k + C. The disk rotates once every P units of time, and its reading head 
passes the start of a new block every L units; so we must have t k = {k-l)L (modulo P). 
Since the processing is sequential, we must also have u k > v k -i for 1 < k < n. There 
are N buffers, hence t k > v k ~ N for IV < k < n. 

How large does N have to be so that the finishing time v n has its minimum possible 
value, T + C + (n — l)ma x(L,C)? Give a general rule for determining the smallest 
such N. Illustrate your rule when L = 1, P = 100, T = .5, n = 100, and (a) C = .5; 
(b) C = 1.0; (c) C = 1.01; (d) C = 1.5; (e) C = 2.0; (f) C = 2.5; (g) C = 10.0; 
(h) C = 50.0; (i) C = 200.0. 
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1.4.5. History and Bibliography 

Most of the fundamental techniques described in Section 1.4 have been developed 
independently by a number of different people, and the exact history of the ideas 
will probably never be known. An attempt has been made to record here the 
most important contributions to the history, and to put them in perspective. 

Subroutines were the first labor-saving devices invented for programmers. 
In the 19th century, Charles Babbage envisioned a library of routines for his 
Analytical Engine [see Charles Babbage and His Calculating Engines, edited by 
Philip and Emily Morrison (Dover, 1961), 56]; and we might say that his dream 
came true in 1944 when Grace M. Hopper wrote a subroutine for computing 
sin x on the Harvard Mark I calculator [see Mechanization of Thought Processes 
(London: Nat. Phys. Lab., 1959), 164]. However, these were essentially “open 
subroutines,” meant to be inserted into a program where needed instead of being 
linked up dynamically. Babbage’s planned machine was controlled by sequences 
of punched cards, as on the Jacquard loom; the Mark I was controlled by a 
number of paper tapes. Thus they were quite different from today’s stored- 
program computers. 

Subroutine linkage appropriate to stored-program machines, with the return 
address supplied as a parameter, was discussed by Herman H. Goldstine and 
John von Neumann in their widely circulated monograph on programming, 
written during 1946 and 1947; see von Neumann’s Collected Works 5 (New York: 
Macmillan, 1963), 215-235. The main routine of their programs was responsible 
for storing parameters into the body of the subroutine, instead of passing the 
necessary information in registers. In England, A. M. Turing had designed 
hardware and software for subroutine linkage as early as 1945; see Proceedings of 
a Second Symposium on Large-Scale Digital Calculating Machinery (Cambridge, 
Mass.: Harvard University, 1949), 87-90; B. E. Carpenter and R. W. Doran, 
editors, A. M. Turing’s ACE Report of 1946 and Other Papers (Cambridge, 
Mass.: MIT Press, 1986), 35-36, 76, 78-79. The use and construction of a 
very versatile subroutine library is the principal topic of the first textbook of 
computer programming, The Preparation of Programs for an Electronic Digital 
Computer, by M. V. Wilkes, D. J. Wheeler, and S. Gill, 1st ed. (Reading, Mass.: 
Addison- Wesley, 1951). 

The word “coroutine” was coined by M. E. Conway in 1958, after he had 
developed the concept, and he first applied it to the construction of an assembly 
program. Coroutines were independently studied by J. Erdwinn and J. Merner, 
at about the same time; they wrote a paper entitled “Bilateral Linkage,” which 
was not then considered sufficiently interesting to merit publication, and un- 
fortunately no copies of that paper seem to exist today. The first published 
explanation of the coroutine concept appeared much later in Conway’s article 
“Design of a Separable Transition-Diagram Compiler,” CACM 6 (1963), 396- 
408. Actually a primitive form of coroutine linkage had already been noted briefly 
as a “programming tip” in an early UNIVAC publication [The Programmer 1,2 
(February 1954), 4]. A suitable notation for coroutines in ALGOL-like languages 
was introduced in Dahl and Nygaard’s SIMULA I [CACM 9 (1966), 671-678], 
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and several excellent examples of coroutines (including replicated coroutines) 
appear in the book Structured Programming by O.-J. Dahl, E. W. Dijkstra, and 
C. A. R. Hoare, Chapter 3. 

The first interpretive routine may be said to be the “Universal Turing 
Machine,” a Turing machine capable of simulating any other Turing machines. 
Turing machines are not actual computers; they are theoretical constructions 
used to prove that certain problems are unsolvable by algorithms. Interpretive 
routines in the conventional sense were mentioned by John Mauchly in his 
lectures at the Moore School in 1946. The most notable early interpreters, 
chiefly intended to provide a convenient means of doing floating point arithmetic' 
were certain routines for the Whirlwind I (by C. W. Adams and others) and 
for the ILLIAC I (by D. J. Wheeler and others). Turing took a part in this 
development also; interpretive systems for the Pilot ACE computer were written 
under his direction. For references to the state of interpreters in the early fifties, 
see the article “Interpretative Sub-routines” by J. M. Bennett. D. G. Prinz 
and M. L. Woods, Proc. ACM (Toronto: 1952), 81-87; see also various papers 
m the Proceedings of the Symposium on Automatic Programming for Digital 
Computers (1954), published by the Office of Naval Research, Washington, D.C. 

The most extensively used early interpretive routine was probably John 
Backus’s “IBM 701 Speedcoding system” [see JACM 1 (1954), 4-6], This 
interpreter was slightly modified and skillfully rewritten for the IBM 650 by 
V. M. Wolontis and others of the Bell Telephone Laboratories; their routine, 
called the “Bell Interpretive System,” became extremely popular. The IPL 
interpretive systems, which were designed beginning in 1956 by A. Newell, J. C. 
Shaw, and H. A. Simon for applications to quite different problems (see Section 
2.6), were used extensively for list processing. Modern uses of interpreters, as 
mentioned in the introduction to Section 1.4.3, are often mentioned in passing 
in the computer literature; see the references listed in that section for articles 
that discuss interpreters in somewhat more detail. 

The first tracing routine was developed by Stanley Gill in 1950: see his 
interesting article in Proceedings of the Royal Society of London , series A, 
206 (1951), 538-554. The text by Wilkes, Wheeler, and Gill mentioned above 
includes several programs for tracing. Perhaps the most interesting of them is 
subroutine C-10 by D. J. Wheeler, which includes a provision for suppressing 
the trace upon entry to a library subroutine, executing the subroutine at full 
speed, then continuing the trace. Published information about trace routines is 
quite rare in the general computer literature, primarily because the methods are 
inherently oriented to a particular machine. The only other early reference known 
to the author is H. V. Meek, “An Experimental Monitoring Routine for the IBM 
705,” Proc. Western Joint Computer Conf. (1956), 68-70, which discusses a trace 
routine for a machine on which the problem was particularly difficult. See also 
the trace routine for IBM’s System/360 architecture, presented in A Compiler 
Generator by W. M. McKeeman, J. J. Horning, and D. B. Wortman (Prentice- 
Hall, 1970), 305-363. Nowadays the emphasis on trace routines has shifted to 
software that provides selective symbolic output and measurements of program 
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performance; one of the best such systems was developed by E. Satterthwaite, 
and described in Software Practice & Experience 2 (1972), 197-217. 

Buffering was originally performed by computer hardware, in a manner 
analogous to the code 1.4.4-(3); an internal buffer area inaccessible to the pro- 
grammer played the role of locations 2000-2099, and the instructions 1.4.4-(3) 
were implicitly performed behind the scenes when an input command was given. 
During the late 1940s, software buffering techniques that are especially useful for 
sorting were developed by early programmers of the UNIVAC (see Section 5.5). 
For a good survey of the prevailing philosophy towards I/O in 1952, see the 
proceedings of the Eastern Joint Computer Conference held in that year. 

The DYSEAC computer [Alan L. Leiner, JACM 1 (1954), 57-81] introduced 
the idea of input-output devices communicating directly with memory while a 
program is running, then interrupting the program upon completion. Such a 
system implies that buffering algorithms had been developed, but the details 
went unpublished. The first published reference to buffering techniques in the 
sense we have described gives a highly sophisticated approach; see O. Mock and 
C. J. Swift, “Programmed Input-Output Buffering,” Proc. ACM Nat. Meeting 
13 (1958), paper 19, and JACM 6 (1959), 145-151. (The reader is cautioned 
that both articles contain a good deal of local jargon, which may take some time 
to understand, but neighboring articles in JACM 6 will help.) An interrupt 
system that enabled buffering of input and output was independently developed 
by E. W. Dijkstra in 1957 and 1958, in connection with B. .1. Loopstra’s and 
C. S. Scholten’s X-l computer [see Comp. J. 2 (1959), 39-43]. Dijkstra’s doc- 
toral thesis, “Communication with an Automatic Computer” (1958), discussed 
buffering techniques, which in this case involved very long circles of buffers since 
the routines were primarily concerned with paper tape and typewriter I/O; 
each buffer contained either a single character or a single number. He later 
developed the ideas into the important general notion of semaphores , which are 
basic to the control of all sorts of concurrent processes, not just input-output 
[see Programming Languages, ed. by F. Genuys (Academic Press, 1968), 43-112; 
BIT 8 (1968), 174-186; Acta Informatica 1 (1971), 115-138], The paper “Input- 
Output Buffering and FORTRAN” by David E. Ferguson, JACM 7 (1960), 1-9, 
describes buffer circles and gives a detailed description of simple buffering with 
many units at once. 


About 1,000 instructions is a reasonable upper limit 
for the complexity of problems now envisioned. 
— HERMAN GOLDSTINE and JOHN VON NEUMANN (1946) 


CHAPTER TWO 


INFORMATION STRUCTURES 

/ think that I shall never see 
A poem lovely as a tree. 
— JOYCE KILMER (1913) 

Yea, from the table of my memory 
I'll wipe away all trivial fond records. 
— HAMLET (Act I, Scene 5, Line 98) 


2.1. INTRODUCTION 

Computer PROGRAMS usually operate on tables of information. In most cases 
these tables are not simply amorphous masses of numerical values; they involve 
important structural relationships between the data elements. 

In its simplest form, a table might be a linear list of elements, when its 
relevant structural properties might include the answers to such questions as: 
Which element is first in the list? Which is last? Which elements precede and 
follow a given one? How many elements are in the list? A lot can be said about 
structure even in this apparently simple case (see Section 2.2). 

In more complicated situations, the table might be a two-dimensional array 
(a matrix or grid, having both a row and a column structure), or it might be 
an n-dimensional array for higher values of n; it might be a tree structure, 
representing hierarchical or branching relationships; or it might be a complex 
multilinked structure with a great many interconnections, such as we may find 
in a human brain. 

In order to use a computer properly, we need to understand the structural 
relationships present within data, as well as the basic techniques for representing 
and manipulating such structure within a computer. 

The present chapter summarizes the most important facts about information 
structures: the static and dynamic properties of different kinds of structure; 
means for storage allocation and representation of structured data; and efficient 
algorithms for creating, altering, accessing, and destroying structural informa- 
tion. In the course of this study, we will also work out several important examples 
that illustrate the application of such methods to a wide variety of problems. 
The examples include topological sorting, polynomial arithmetic, discrete system 
simulation, sparse matrix transformation, algebraic formula manipulation, and 
applications to the writing of compilers and operating systems. Our concern 
will be almost, entirely with structure as represented inside a computer; the 
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conversion from external to internal representations is the subject of Chapters 9 
and 10. 

Much of the material we will discuss is often called “List processing,” since 
a number of programming systems such as LISP have been designed to facilitate 
working with general kinds of structures called Lists. (When the word “list” 
is capitalized in this chapter, it is being used in a technical sense to denote 
a particular type of structure that is highlighted in Section 2.3.5.) Although 
List processing systems are useful in a large number of situations, they impose 
constraints on the programmer that are often unnecessary; it is usually better to 
use the methods of this chapter directly in one’s own programs, tailoring the data 
format and the processing algorithms to the particular application. Many people 
unfortunately still feel that List processing techniques are quite complicated (so 
that it is necessary to use someone else’s carefully written interpretive system or 
a prefabricated set of subroutines), and that List processing must be done only 
in a certain fixed way. We will see that there is nothing magic, mysterious, or 
difficult about the methods for dealing with complex structures; these techniques 
are an important part of every programmer’s repertoire, and we can use them 
easily whether we are writing a program in assembly language or in an algebraic 
language like FORTRAN, C, or Java. 

We will illustrate methods of dealing with information structures in terms 
of the MIX computer. A reader who does not care to look through detailed 
MIX programs should at least study the ways in which structural information is 
represented in Mix’s memory. 

It is important at this point to define several terms and notations that we 
will be using frequently from now on. The information in a table consists of a 
set of nodes (called “records,” “entities,” or “beads” by some authors); we will 
occasionally say “item” or “element” instead of “node.” Each node consists of 
one or more consecutive words of the computer memory, divided into named 
parts called fields. In the simplest case, a node is just one word of memory, and 
it has just one field comprising that whole word. As a more interesting example, 
suppose the elements of our table are intended to represent playing cards; we 
might have two-word nodes broken into five fields, TAG, SUIT, RANK, NEXT, and 
TITLE: 

(i) 


(This format reflects the contents of two MIX words. Recall that a MIX word 
consists of five bytes and a sign; see Section 1.3.1. In this example we assume 
that the signs are + in each word.) The address of a node, also called a link, 
pointer, or reference to that node, is the memory location of its first word. The 
address is often taken relative to some base location, but in this chapter for 
simplicity we will take the address to be an absolute memory location. 

The contents of any field within a node may represent numbers, alphabetic 
characters, links, or anything else the programmer may desire. In connection 
with the example above, we might wish to represent a pile of cards that might 


+ 

TAG 

SUIT 

RANK 

NEXT 

+ 



TITLE 
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appear m a game of solitaire: TAG = 1 means that the card is face down TAG = 0 
means that it is face up; SUIT = 1, 2, 3, or 4 for clubs, diamonds, hearts, or 

spades, respectively, RANK = 1, 2 13 for ace, deuce, . . . , king; NEXT is a link 

to the card below this one m the pile; and TITLE is a five-character alphabetic 
name of this card, for use m printouts. A typical pile might look like this: 


Actual cards 


Computer representation 




The memory locations in the computer representation are shown here as 100, 
3Sb, and 242; they could have been any other numbers as far as this example 
is concerned, since each card links to the one below it. Notice the special link 
A m node 100; we use the capital Greek letter Lambda to denote the null link, 
le link to no node. The null link A appears in node 100 since the 10 of clubs 
is the bottom card of the pile. Within the machine, A is represented by some 
easily recognizable value that cannot be the address of a node. We will generally 
assume that no node appears in location 0; consequently, A will almost always 
be represented as the link value 0 in MIX programs. 

The introduction of links to other elements of data is an extremely important 
idea m computer programming; links are the key to the representation of complex 
structures. When displaying computer representations of nodes it is usually 
convenient to represent links by arrows, so that example (2) would appear thus: 
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U 
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0 | 
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u , c 


(3) 


The actual locations 242, 386, and 100 (which are irrelevant anyway) no longer 
appear m representation (3). Electrical circuit notation for a “grounded” wire 
is used to indicate a null link, shown here at the right of the diagram. Notice 
also that (3) indicates the top card by an arrow from “TOP”; here TOP is a link 
variable, often called a pointer variable, namely a variable whose value is a link. 

references to nodes m a program are made directly through link variables (or 
link constants), or indirectly through link fields in other nodes. 
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Now we come to the most important part of the notation, the means of 
referring to fields within nodes. This is done simply by giving the name of the 
field followed by a link to the desired node in parentheses; for example, in ( 2 ) 
and ( 3 ) with the fields of ( 1 ) we have 

RANK (100) = 10; SUIT(TOP) * 2; 

(4 

TITLE(TOP) = “ U u2 u D”; RANK (NEXT (TOP) ) =3. 

The reader should study these examples carefully, since such field notations 
will be used in many algorithms of this chapter and the following chapters. To 
make the ideas clearer, we will now state a simple algorithm for placing a new 
card face up on top of the pile, assuming that NEWCARD is a link variable whose 
value is a link to the new card: 

Al. Set NEXT (NEWCARD) <— TOP. (This puts the appropriate link into the new 
card node.) 

A2. Set TOP e- NEWCARD. (This keeps TOP pointing to the top of the pile.) 

A3. Set TAG (TOP) 0. (This marks the card as “face up.”) | 

Another example is the following algorithm, which counts the number of cards 
currently in the pile: 

Bl. Set N 0, X <— TOP. (Here N is an integer variable, X is a link variable.) 

B2. If X = A, stop; N is the number of cards in the pile. 

B3. Set N <— N + 1, X <— NEXT(X) , and go back to step B2. | 

Notice that we are using symbolic names for two quite different things in 
these algorithms: as names of variables (TOP, NEWCARD, N, X) and as names of 
fields (TAG, NEXT). These two usages must not be confused. If F is a field name 
and L / A is a link, then F(L) is a variable; but F itself is not a variable it 
does not possess a value unless it is qualified by a nonnull link. 

Two further notations are used, to convert between addresses and the values 
stored there, when we are discussing low-level machine details: 

a) CONTENTS always denotes a full-word field of a one-word node. Thus 
CONTENTS (1000) denotes the value stored in memory location 1000; it is a 
variable having this value. If V is a link variable, CONTENTS (V) denotes the 
value pointed to by V (not the value V itself). 

b) If V is the name of some value held in a memory cell, L0C(V) denotes the 
address of that cell. Consequently, if V is a variable whose value is stored in a 
full word of memory, we have CONTENTS (L0C (V) ) = V. 

It is easy to transform this notation into MIXAL assembly language code, 
although MIXAL's notation is somewhat backwards. The values of link variables 
are put into index registers, and the partial-field capability of MIX is used to refer 
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to the desired field. For example, Algorithm A above could be written thus: 


NEXT 

EQU 

4:5 

Definition of the NEXT 

TAG 

EQU 

1:1 

and TAG fields for the assembler 


LD1 

NEWCARD 

Al. rll NEWCARD. 


LDA 

TOP 

rA <- TOP. 


STA 

0,1 (NEXT) 

NEXT(rll) <- rA. 


ST1 

TOP 

A2. TOP <- rll. 


STZ 

0 , 1 (TAG) 

AT TAG(rll) <— 0. | 


The ease and efficiency with which these operations can be carried out in a 
computer is the primary reason for the importance of the “linked memory” 
concept. 

Sometimes we have a single variable that denotes a whole node; its value is 
a sequence of fields instead of just one field. Thus we might write 

CARD NODE (TOP), (6) 

where NODE is a field specification just like CONTENTS, except that it refers to an 
entire node, and where CARD is a variable that assumes structured values like 
those in (i). If there are c words in a node, the notation (6) is an abbreviation 
for the c low-level assignments 

CONTENTS (LOC (CARD) + j) <- CONTENTS (TOP + j) , 0 < j < c. (7) 

There is an important distinction between assembly language and the no- 
tation used in algorithms. Since assembly language is close to the machine’s 
internal language, the symbols used in MIXAL programs stand for addresses 
instead of values. Thus in the left-hand columns of (5), the symbol TOP actually 
denotes the address where the pointer to the top card appears in memory; but 
m (6) and (7) and in the remarks at the right of (5), it denotes the value of 
TOP, namely the address of the top card node. This difference between assembly 
language and higher-level language is a frequent source of confusion for beginning 
programmers, so the reader is urged to work exercise 7. The other exercises also 
provide useful drills on the notational conventions introduced in this section. 

EXERCISES 

1. [04] In the situation depicted in (3), what is the value of (a) SUIT(NEXT(TOP) ) ■ 
(b) NEXT (NEXT (NEXT (TOP))) ? 

2. [10] The text points out that in many cases CONTENTS (LOC (V) ) = V. Under what 
conditions do we have LOC (CONTENTS (V) ) = V? 

3. [11] Give an algorithm that essentially undoes the effect of Algorithm A: It 
removes the top card of the pile (if the pile is not empty) and sets NEWCARD to the 
address of this card. 

4. [18] Give an algorithm analogous to Algorithm A, except that it puts the new 
card face down at the bottom of the pile. (The pile may be empty.) 

► 5. [21] Give an algorithm that essentially undoes the effect of exercise 4: Assuming 
that the pile is not empty and that its bottom card is face down, your algorithm should 
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remove the bottom card and make NEWCARD link to it. (This algorithm is sometimes 
called “cheating” in solitaire games.) 

6 . [06'] In the playing card example, suppose that CARD is the name of a variable 
whose value is an entire node as in ( 6 ). The operation CARD «- NODE(TOP) sets the 
fields of CARD respectively equal to those of the top of the pile. After this operation, 
which of the following notations stands for the suit of the top card? (a) SUIT (CARD); 
(b) SUIT (LOC(CARD) ) ; (c) SUIT (CONTENTS (CARD) ) ; (d) SUIT(TOP) ? 

► 7 . [ 04 ] In the text’s example MIX program, ( 5 ), the link variable TOP is stored in the 
MIX computer word whose assembly language name is TOP. Given the field structure ( 1 ), 
which of the following sequences of code brings the quantity NEXT(TOP) into register A? 
Explain why the other sequence is incorrect. 

a) LDA TOP (NEXT) b) LD1 TOP 

LDA 0,1 (NEXT) 

► 8 . [1<S] Write a MIX program corresponding to steps B1-B3. 

9. [ 23 ] Write a MIX program that prints out the alphabetic names of the current 
contents of the card pile, starting at the top card, with one card per line, and with 
parentheses around cards that are face down. 
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2.2. LINEAR LISTS 

2.2.1. Stacks, Queues, and Deques 

Data USUALLY has much more structural information than we actually want to 
represent directly in a computer. For example, each “playing card” node in the 
preceding section had a NEXT field to specify what card was beneath it in the pile, 
but we provided no direct way to find what card, if any, was above a given card, 
or to find what pile a given card was in. And of course we totally suppressed 
most of the characteristic features of real playing cards: the details of the design 
on the back, the relation to other objects in the room where the game was being- 
played, the individual molecules within the cards, etc. It is conceivable that such 
structural information would be relevant in certain computer applications, but 
obviously we never want to store all of the structure that is present in every 
situation. Indeed, for most card-playing situations we would not need all of the 
facts retained in the earlier example; the TAG field, which tells whether a card is 
face up or face down, will often be unnecessary. 

We must decide in each case how much structure to represent in our tables, 
and how accessible to make each piece of information. To make such decisions, 
we need to know what operations are to be performed on the data. For each 
problem considered in this chapter, therefore, we consider not only the data 
structure but also the class of operations to be done on the data ; the design of 
computer representations depends on the desired function of the data as well as 
on its intrinsic properties. Indeed, an emphasis on function as well as form is 
basic to design problems in general. 

In order to illustrate this point further, let’s consider a related aspect of 
computer hardware design. A computer memory is often classified as a “random 
access memory,” like Mix’s main memory; or as a “read-only memory,” which 
is supposed to contain essentially constant information; or a “secondary bulk 
memory,” like Mix’s disk units, which cannot be accessed at high speed al- 
though large quantities of information can be stored; or an “associative memory,” 
more properly called a “content-addressed memory,” for which information is 
addressed by its value rather than by its location; and so on. The intended 
function of each kind of memory is so important that it enters into the name of 
the particular memory type; all of these devices are “memory” units, but the 
purposes to which they are put profoundly influence their design and their cost. 

A linear list is a sequence of n > 0 nodes X [1] , X [2] , ..., X [rc] whose 
essential structural properties involve only the relative positions between items 
as they appear in a line. The only things we care about in such structures are the 
facts that, if n > 0, X [1] is the first node and X [n] is the last; and if 1 < fc < n, 
the fcth node X[fc] is preceded by X[fc - 1] and followed by X[fc + 1] . 

The operations we might want to perform on linear lists include, for example, 
the following. 

i) Gain access to the fcth node of the list to examine and/or to change the 

contents of its fields. 
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ii) Insert a new node just before or after the fcth node. 

iii) Delete the A;th node. 

iv) Combine two or more linear lists into a single list. 

v) Split a linear list into two or more lists. 

vi) Make a copy of a linear list. 

vii) Determine the number of nodes in a list. 

viii) Sort the nodes of the list into ascending order based on certain fields of the 
nodes. 

ix) Search the list for the occurrence of a node with a particular value in some 
field. 

In operations (i), (ii), and (iii) the special cases k = 1 and k — n are of principal 
importance, since the first and last items of a linear list may be easier to get at 
than a general element is. We will not discuss operations (viii) and (ix) in this 
chapter, since those topics are the subjects of Chapters 5 and 6, respectively. 

A computer application rarely calls for all nine of these operations in their 
full generality, so we find that there are many ways to represent linear lists 
depending on the class of operations that are to be done most frequently. It 
is difficult to design a single representation method for linear lists in which all 
of these operations are efficient; for example, the ability to gain access to the 
fcth node of a long list for random k is comparatively hard to do if at the same 
time we are inserting and deleting items in the middle of the list. Therefore we 
distinguish between types of linear lists depending on the principal operations to 
be performed, just as we have noted that computer memories are distinguished 
by their intended applications. 

Linear lists in which insertions, deletions, and accesses to values occur almost 
always at the first or the last node are very frequently encountered, and we give 
them special names: 

A stack is a linear list for which all insertions and deletions (and usually all 
accesses) are made at one end of the list. 

A queue is a linear list for which all insertions are made at one end of the 
list; all deletions (and usually all accesses) are made at the other end. 

A deque (‘"double-ended queue”) is a linear list for which all insertions and 
deletions (and usually all accesses) are made at the ends of the list. 

A deque is therefore more general than a stack or a queue; it has some properties 
in common with a deck of cards, and it is pronounced the same way. We also 
distinguish output-restricted or input-restricted deques, in which deletions or 
insertions, respectively, are allowed to take place at only one end. 

In some disciplines the word “queue” has been used in a much broader sense, 
to describe any kind of list that is subject to insertions and deletions; the special 
cases identified above are then called various “queuing disciplines.” Only the 
restricted use of the term “queue” is intended in this book, however, by analogy 
with orderly queues of people waiting in line for service. 
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Fig. 1. A stack represented as a rail- 
way switching network. 


Output from stack Input to stack 



Sometimes it helps to understand the mechanism of a stack in terms of an 
analogy from the switching of railroad cars, as suggested by E. W. Dijkst.ra (see 
Fig. 1). A corresponding picture for deques is shown in Fig. 2. 


This track closed on input-restricted deque 



Fig. 2. A deque represented as a railway switching network. 


With a stack we always remove the “youngest” item currently in the list, 
namely the one that has been inserted more recently than any other. With a 
queue just the opposite is true: The “oldest” item is always removed: the nodes 
leave the list in the same order as they entered it. 

Many people who have independently realized the importance of stacks and 
queues have given them other names: Stacks have been called push-down lists, 
reversion storages, cellars, nesting stores, piles, last-in-first-out (“LIFO”) lists, 
and even yo-yo lists. Queues are sometimes called circular stores or first-in-first- 
out (“FIFO”) lists. The terms LIFO and FIFO have been used for many years 
by accountants, as names of methods for pricing inventories. Still another term, 
“shelf,” has been applied to output-restricted deques, and input-restricted deques 
have been called scrolls or “rolls. This multiplicity of names is interesting in 
itself, since it is evidence for the importance of the concepts. The words stack 
and queue are gradually becoming standard terminology; of all the other words 
listed above, only “push-down list” is still reasonably common, particularly in 
connection with automata theory. 

Stacks arise quite frequently in practice. We might, for example, go through 
a set of data and keep a list of exceptional conditions or things to do later; after 
we re done with the original set, we can then do the rest of the processing by 
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coming back to the list, removing entries until it becomes empty. (The “saddle 
point” problem, exercise 1.3.2-10, is an instance of this situation.) Either a stack 
or a queue will be suitable for such a list, but a stack is generally more convenient. 
We all have “stacks” in our minds when we are solving problems: One problem 
leads to another and this leads to another; we stack up problems and subproblems 
and remove them as they are solved. Similarly, the process of entering and 
leaving subroutines during the execution of a computer program has a stack-like 
behavior. Stacks are particularly useful for the processing of languages with a 
nested structure, like programming languages, arithmetic expressions, and the 
literary German “Schachtelsatze.” In general, stacks occur most frequently in 
connection with explicitly or implicitly recursive algorithms, and we will discuss 
this connection thoroughly in Chapter 8. 



(a) Stack 


Delete Insert 



Front Second Third Rear 

(b) Queue 


Second Second 

Leftmost from left from right Rightmost 



Insert or 
delete 


(c) Deque 


Insert or 
delete 


Fig. 3. Three important classes of linear lists. 


Special terminology is generally used when algorithms refer to these struc- 
tures: We put an item onto the top of a stack, or take the top item off (see 
Fig. 3a). Hie bottom of the stack is the least accessible item, and it will not be 
removed until all other items have been deleted. (People often say that they push 
an item down onto a stack, and pop the stack up when the top item is deleted. 
This terminology comes from an analogy with the stacks of plates often found in 
cafeterias. The brevity of the words “push” and “pop” has its advantages, but 
these terms falsely imply a motion of the whole list within computer memory. 
Nothing is physically pushed down; items are added onto the top, as in haystacks 
or stacks of boxes.) With queues, we speak of the front and the rear of the queue; 
things enter at the rear and are removed when they ultimately reach the front po- 
sition (see Fig. 3b). When referring to deques, we speak of the left and right ends 
(Fig. 3c). The concepts of top, bottom, front, and rear are sometimes applied 
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to deques that are being used as stacks or queues, with no standard conventions 
as to whether top, front, and rear should appear at the left or the right. 

Thus we find it easy to use a rich variety of descriptive words from English in 
our algorithms: “up-down” terminology for stacks, “waiting in line” terminology 
for queues, and “left-right” terminology for deques. 

A little bit of additional notation has proved to be convenient for dealing 
with stacks and queues: We write 

A <= x (i) 

(when A is a stack) to mean that the value x is inserted on top of stack A, 
or (when A is a queue) to mean that x is inserted at the rear of the queue. 
Similarly, the notation 

x 4= A (2) 

is used to mean that the variable x is set equal to the value at the top of stack A 
or at the front of queue A, and this value is deleted from A. Notation (2) is 
meaningless when A is empty — that is, when A contains no values. 

If A is a nonempty stack, we may write 

top(A) (3) 

to denote its top element. 

EXERCISES 

1. [06] An input-restricted deque is a linear list in which items may be inserted at 
one end but removed from either end; clearly an input-restricted deque can operate 
either as a stack or as a queue, if we consistently remove all items from one of the two 
ends. Can an output-restricted deque also be operated either as a stack or as a queue? 

► 2. [15] Imagine four railroad cars positioned on the input side of the track in Fig. 1. 

numbered 1, 2, 3, and 4, from left to right. Suppose we perform the following sequence 
of operations (which is compatible with the direction of the arrows in the diagram and 
does not require cars to “jump over” other cars): (i) move car 1 into the stack; (ii) move 
car 2 into the stack; (iii) move car 2 into the output; (iv) move car 3 into the stack; 
(v) move car 4 into the stack; (vi) move car 4 into the output; (vii) move car 3 into 
the output; (viii) move car 1 into the output. 

As a result of these operations the original order of the cars, 1234, has been 
changed into 2431. It is the purpose of this exercise and the following exercises to 
examine what permutations are obtainable in such a manner from stacks, queues, or 
deques. 

If there are six railroad cars numbered 123456, can they be permuted into the 
order 325641? Can they be permuted into the order 154623? (In case it is possible, 
show how to do it.) 

3. [25] The operations (i) through (viii) in the previous exercise can be much more 
concisely described by the code SSXSSXXX, where S stands for “move a car from the 
input into the stack,” and X stands for “move a car from the stack into the output.” 
Some sequences of S’s and X’s specify meaningless operations, since there may be no 
cars available on the specified track; for example, the sequence SXXSSXXS cannot be 
carried out, since we assume that the stack is initially empty. 
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Let us call a sequence of S’s and X’s admissible if it contains n S’s and n X’s, and 
if it specifies no operations that cannot be performed. Formulate a rule by which it is 
easy to distinguish between admissible and inadmissible sequences; show furthermore 
that no two different admissible sequences give the same output permutation. 

4. [MS 4] Find a simple formula for a n , the number of permutations on n elements 
that can be obtained with a stack like that in exercise 2. 

► 5. [ M28 ] Show that it is possible to obtain a permutation P1P2 ■ ■ ■ p n from 12 . . . n 
using a stack if and only if there are no indices i < j < k such that pj < p k < p t . 

6. [00] Consider the problem of exercise 2, with a queue substituted for a stack. 
What permutations of 12 ... n can be obtained with use of a queue? 

► 7. [25] Consider the problem of exercise 2, with a deque substituted for a stack, 
(a) Find a permutation of 1234 that can be obtained with an input-restricted deque, 
but it cannot be obtained with an output-restricted deque, (b) Find a permutation 
of 1234 that can be obtained with an output-restricted deque but not with an input- 
restricted deque. [As a consequence of (a) and (b), there is definitely a difference 
between input-restricted and output-restricted deques.] (c) Find a permutation of 1234 
that cannot be obtained with either an input-restricted or an output-restricted deque. 

8. [22] Are there any permutations of 12 ... n that cannot be obtained with the use 
of a deque that is neither input- nor output-restricted? 

9. [M20] Let b n be the number of permutations on n elements obtainable by the use 
of an input-restricted deque. (Note that b 4 = 22, as shown in exercise 7.) Show that 
b n is also the number of permutations on n elements with an output-restricted deque. 

10. [M25] (See exercise 3.) Let S, Q, and X denote respectively the operations of in- 
serting an element at the left, inserting an element at the right, and emitting an element 
from the left, of an output-restricted deque. For example, the sequence QQXSXSXX 
will transform the input sequence 1234 into 1342. The sequence SXQSXSXX gives the 
same transformation. 

Find a way to define the concept of an admissible sequence of the symbols S, Q, 
and X, so that the following property holds: Every permutation of n elements that 
is attainable with an output-restricted deque corresponds to precisely one admissible 
sequence. 

► 11 . [M 40 ] As a consequence of exercises 9 and 10, the number b n is the number of 
admissible sequences of length 2 n. Find a closed form for the generating function 

5Zn>o b n z n . 

12. [HM34] Compute the asymptotic values of the quantities a n and b n in exercises 
4 and 11. 

13. [MJ,8] How many permutations of n elements are obtainable with the use of a 
general deque? [See Rosenstiehl and Tarjan, J. Algorithms 5 (1984), 389-390, for an 
algorithm that decides in O(n) steps whether or not a given permutation is obtainable.] 

► 14. [26] Suppose you are allowed to use only stacks as data structures. How can you 
implement a queue efficiently with two stacks? 
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2.2.2. Sequential Allocation 

The simplest and most natural way to keep a linear list inside a computer is to 
put the list items in consecutive locations, one node after the other. Then we 
will have 

L0C(X[j + 1] ) =L0C(X[j]) +c, 

where c is the number of words per node. (Usually c = 1 . When c > 1 , it is 
sometimes more convenient to split a single list into c “parallel” lists, so that 
the fcth word of node X [j] is stored a fixed distance from the location of the 
first word of X[j], depending on k. We will continually assume, however, that 
adjacent groups of c words form a single node.) In general. 

L0C(X[j]) = Lo + cj, (i) 

where L 0 is a constant called the base address, the location of an artificially 
assumed node X [0] . 

This technique for representing a linear list is so obvious and well-known 
that there seems to be no need to dwell on it at any length. But we will be 
seeing many other “more sophisticated” methods of representation later on in 
this chapter, and it is a good idea to examine the simple case first to see just 
how far we can go with it. It is important to understand the limitations as well 
as the power of the use of sequential allocation. 

Sequential allocation is quite convenient for dealing with a stack. We simply 
have a variable T called the stack pointer. When the stack is empty, we let T = 0. 
To place a new element Y on top of the stack, we set 

T+-T + 1; X [T] <— Y. ( 2 ) 

And when the stack is not empty, we can set Y equal to the top node and delete 
that node by reversing the actions of ( 2 ): 

Y <— X [T] ; T ■<— T — 1. ( 3 ) 

(Inside a computer it is usually most efficient to maintain the value cT instead 
of T, because of ( 1 ). Such modifications are easily made, so we will continue our 
discussion as though c = 1 . ) 

The representation of a queue or a more general deque is a little trickier. An 
obvious solution is to keep two pointers, say F and R (for the front and rear of 
the queue), with F = R = 0 when the queue is empty. Then inserting an element 
at the rear of the queue would be 

R R + 1; X [R] y— Y. ( 4 ) 

Removing the front node (F points just below the front) would be 

F F + 1; Y X[F] ; if F = R. then set F R 0. ( 5 ) 

But note what can happen: If R always stays ahead of F (so that there is 
always at least one node in the queue) the table entries used are X [1] . X [2] . 

• • • , X [ 1000 ] , . . . , ad infinitum, and this is terribly wastefid of storage space. 
The simple method of ( 4 ) and ( 5 ) should therefore be used only in the situation 


2 . 2.2 


SEQUENTIAL ALLOCATION 245 


when F is known to catch up to R quite regularly — for example, if all deletions 
come in spurts that empty the queue. 

To circumvent the problem of the queue overrunning memory, we can set 
aside M nodes X [1] , .... X [M] arranged implicitly in a circle with X [1] following 
X [M] . Then processes (4) and (5) above become 

if R = M then R 4— 1, otherwise R 4— R + 1; X[R] 4— Y. (6) 

if F = M then F 4— 1, otherwise F 4— F + 1; Y 4— X[F] . (7) 

We have, in fact, already seen circular queuing action like this, when we looked 

at input-output buffering in Section 1.4.4. 

Our discussion so far has been very unrealistic, because we have tacitly 
assumed that nothing could go wrong. When we deleted a node from a stack or 
queue, we assumed that there was at least one node present. When we inserted 
a node into a stack or queue, we assumed that there was room for it in memory. 
But clearly the method of (6) and (7) allows at most M nodes in the entire queue, 
and methods (2), (3), (4), (5) allow T and R to reach only a certain maximum 
amount within any given computer program. The following specifications show 
how the actions should be rewritten for the common case where we do not assume 
that these restrictions are automatically satisfied: 

r T 4- T + 1; 

X 4= Y (insert into stack): < if T > M, then OVERFLOW: (2a) 

l X[T] 4- Y. 

r if T = 0, then UNDERFLOW; 

Y 4= X (delete from stack): < Y 4— X[T] ; (3a) 

l T 4— T — 1. 

{ if R = M, then R 4— 1, otherwise R 4— R + 1; 
if R = F, then OVERFLOW; (6a) 

X [R] 4 — Y. 

( if F = R, then UNDERFLOW: 

Y X (delete from queue): < if F = M, then F 4— 1, otherwise F 4— F + 1; (7a) 

l Y 4— X [F] . 

Here we assume that X [1] , . . . , X [M] is the total amount of space allowed for the 
list; OVERFLOW and UNDERFLOW mean an excess or deficiency of items. The initial 
setting F = R = 0 for the queue pointers is no longer valid when we use (6a) and 
(7a), because overflow will not be detected when F = 0; we should start with 
F R — 1, say. 

The reader is urged to work exercise 1, which discusses a nontrivial aspect 
of this simple queuing mechanism. 

The next question is, “What do we do when UNDERFLOW or OVERFLOW oc- 
curs?” I11 the case of UNDERFLOW, we have tried to remove a nonexistent item; 
this is usually a meaningful condition — not an error situation — that can be 
used to govern the flow of a program. For example, we might want to delete 
items repeatedly until UNDERFLOW occurs. An OVERFLOW situation, however, is 
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usually an error; it means that the table is full already, yet there is still more 
information waiting to be put in. The usual policy in case of OVERFLOW is to 
report reluctantly that the program cannot go on because its storage capacity 
has been exceeded; then the program terminates. 

Of course we hate to give up in an OVERFLOW situation when only one list 
has gotten too large, while other lists of the same program may very well have 
plenty of room remaining. In the discussion above we were primarily thinking of 
a program with only one list. However, we frequently encounter programs that 
involve several stacks, each of which has a dynamically varying size. In such a 
situation we don’t want to impose a maximum size on each stack, since the size 
is usually unpredictable; and even if a maximum size has been determined for 
each stack, we will rarely find all stacks simultaneously filling their maximum 
capacity. 

When there are just two variable-size lists, they can coexist together very 
nicely if we let the lists grow toward each other: 


Program and 
fixed-size tables 


List 1 



Available 

space 


List 2 


Program and 
fixed-size tables 


Beginning 
of memory 


Bottom 


t 

Top 


t 

Top 


• '■ V 



Bottom 


End of 
memory 


Here list 1 expands to the right, and list 2 (stored in reverse order) expands to 
the left. OVERFLOW will not occur unless the total size of both lists exhausts all 
memory space. The lists may independently expand and contract so that the 
effective maximum size of each one could be significantly more than half of the 
available space. This layout of memory space is used very frequently. 

We can easily convince ourselves, however, that there is no way to store three 
or more variable-size sequential lists in memory so that (a) OVERFLOW will occur 
only when the total size of all lists exceeds the total space, and (b) each list has 
a fixed location foi its bottom ’ element. When there are, say, ten or more 
variable-size lists and this is not unusual — the storage allocation problem 
becomes very significant. If we wish to satisfy condition (a), we must give up 
condition (b); that is, we must allow the “bottom” elements of the lists to change 
tlieii positions. This means that the location Lq of Eq. ( 1 ) is not constant any 
longer, no reference to the table may be made to an absolute memory address, 
since all references must be relative to the base address L 0 . In the case of MIX, 
the coding to bring the Ith one- word node into register A is changed from 


LD1 I 
LDA L 0 , 1 


to, for example, 


LD1 I 

LDA BASE(0 : 2) 
STA *+1(0:2) 
LDA * , 1 


( 8 ) 


where BASE contains | L 0 ’ 1 0 1 0 ] 0 | . Such relative addressing evidently 
takes longer than fixed-base addressing, although it would be only slightly slower 
if MIX had an “indirect addressing” feature (see exercise 3). 
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An important special case occurs when each of the variable-size lists is a 
stack. Then, since only the top element of each stack is relevant at any time, we 
can proceed almost as efficiently as before. Suppose that we have n stacks; the 
insertion and deletion algorithms above become the following, if BASE[i] and 
TOP [i] are link variables for the Ah stack, and if each node is one word long: 


Insertion: T0P[i] 4— T0P[*] +1; ifT0P[i] > BASE[i + 1] , then 

OVERFLOW; otherwise set CONTENTS (TOP [i] ) 4— Y. (9) 

Deletion: if TOP [i] = BASE [i] , then UNDERFLOW ; otherwise 

set Y 4- CONTENTS (TOP [i] ) , TOP [t] 4- TOP [i] - 1. (10) 


Here BASE[t + 1] is the base location of the ( i + l)st stack. The condition 
TOP [i] = BASE [i] means that stack i is empty. 

In (9), OVERFLOW is no longer such a crisis as it was before; we can “repack 
memory,” making room for the table that overflowed by taking some away from 
tables that aren’t yet filled. Several ways to do the repacking suggest themselves; 
we will now consider some of them in detail, since they can be quite important 
when linear lists are allocated sequentially. We will start by giving the simplest 
of the methods, and will then consider some of the alternatives. 

Assume that there are n stacks, and that the values BASE[i] and T0P[i] 
are to be treated as in (9) and (10). These stacks are all supposed to share a 
common memory area consisting of all locations L with Lo < L < Loo. (Here Lo 
and Loo are constants that specify the total number of words available for use.) 
We might start out with all stacks empty, and 


BASE [j] = TOP [j] = L 0 for 1 < j < n. (11) 


We also set BASE [n + 1] = Loo so that (9) will work properly for i — n. 

When OVERFLOW occurs with respect to stack i, there are three possibilities: 

a) We find the smallest A for which i < k < n and TOP [A;] < BASE [A- + 1], if 
any such k exist. Now move things up one notch: 

Set CONTENTS (L + 1) 4- CONTENTS (L) , for TOP [A] > L > BASE [i + 1] . 

(This must be done for decreasing, not increasing, values of L to avoid losing 
information. It is possible that TOP [A] = BASE Li + 1] , in which case nothing 
needs to be moved.) Finally we set BASE[j] 4— BASE[j] + 1 and T0P[j] 4— 
T0P[j] + 1, for i < j < A. 

b) No A can be found as in (a), but we find the largest A for which 1 < A < i 
and TOP [A] < BASE [A + 1] . Now move things down one notch: 

Set CONTENTS (L - 1) 4- CONTENTS (L), for BASE [A + 1] < L < T0P[i]. 

(This must be done for increasing values of L.) Then set BASE[j] 4— 
BASE [j] - 1 and TOP [ j] 4- T0P[j] - 1, for A < j < i. 

c) We have TOP [A] = BASE [A + 1] for all A / i. Then obviously we cannot 
find room for the new stack entry, and we must give up. 
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Fig. 4. Example of memory configuration after several insertions and deletions. 


Figure 4 illustrates the configuration of memory for the case n = 4, L 0 = 0, 
Lqo = 20 , after the successive actions 


it il i 4 i;d x i* 1, it 1; i 4 d 2 Dj . 


( 12 ) 


(Here lj and D, refer to insertion and deletion in stack j, and an asterisk refers 
to an occurrence of OVERFLOW, assuming that no space is initially allocated to 
stacks 1, 2, and 3.) 


It is clear that many of the first stack overflows that occur with this method 
could be eliminated if we chose our initial conditions wisely, instead of allocating 
all space initially to the nth stack as suggested in ( 11 ). For example, if we expect 
each stack to be of the same size, we can start out with 


BASE [j] = TOP [j] = 


3-1 


^ (Loo Lq) 


for 1 < j < n. 


(13) 


Operating experience with a particular program may suggest better starting 
values; however, no matter how well the initial allocation is set up, it can save 
at most a fixed number of overflows, and the effect is noticeable only in the early 
stages of a program run. (See exercise 17.) 

Another possible way to improve the method above would be to make room 
for more than one new entry each time memory is repacked. This idea has 
been exploited by J. Garwick, who suggests a complete repacking of memory 
when overflow occurs, based on the change in size of each stack since the last 
repacking. His algorithm uses an additional array, called 0LDT0P[ j] f < j < n 
which retains the value that TOP [j ] had just after the previous allocation of 
memory. Initially, the tables are set as before, with 0LDT0P [ j] = Top [ j ] . The 
algorithm proceeds as follows: 


Algorithm G ( Reallocate sequential tables). Assume that OVERFLOW has oc- 
curred in stack i, according to ( 9 ). After Algorithm G has been performed, 
either we will find the memory capacity exceeded or the memory will have been 
rearranged so that the action NODE (TOP [t] ) <- Y may be done. (Notice that 
TO? [*] h as already been increased in ( 9 ) before Algorithm G takes place.) 

Gl. [Initialize.] Set SUM <- Loo - L 0 , INC <- 0. Then do step G2 for 1 <j< n . 
(The effect will be to make SUM equal to the total amount of memory space 
left, and INC equal to the total amount of increases in table sizes since the 
last allocation.) After this has been done, go on to step G3. 
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G2. [Gather statistics.] Set SUM 4- SUM - (T0P[j] - BASE [j] ). If T0P[j] > 
OLDTOP [ j] , set D [ j] 4- TOP [ jl - OLDTOP [ j] and INC 4 — INC + D [j] ; 

otherwise set D [ j] 4— 0. 

G3. [Is memory full?] If SUM < 0, we cannot proceed. 

G4. [Compute allocation factors.] Set a 4— 0.1 X SUM/n, (3 4— 0.9 x SUM/INC. 
(Here a and f3 are fractions, not integers, which are to be computed to 
reasonable accuracy. The following step awards the available space to indi- 
vidual lists as follows: Approximately 10 percent of the memory presently 
available will be shared equally among the n lists, and the other 90 percent 
will be divided proportionally to the amount of increase in table size since 
the previous allocation.) 

G5. [Compute new base addresses.] Set NEWBASE[1] 4— BASE[1] and a 4— 
0; then for j = 2, 3, . . . , n set r 4— a + a + D [ j — 1] /3, NEWBASE [j] 4— 
NEWBASE [j - 1] + TOP [j - 1] - BASE [j - 1] + [rj - [erj, and cr 4- r. 

G6. [Repack.] Set TOP [z] 4— T0P[i] — 1. (This reflects the true size of the ith 
list, so that no attempt will be made to move information from beyond 
the list boundary.) Perform Algorithm R below, and then reset T0P[/] 4— 
TOP [/] + 1. Finally set OLDTOP [j] 4- T0P[j] for 1 < j < n. \ 

Perhaps the most interesting part of this whole algorithm is the general 
repacking process, which we shall now describe. Repacking is not trivial, since 
some portions of memory shift up and others shift down; it is obviously important 
not to overwrite any of the good information in memory while it is being moved. 

Algorithm R ( Relocate sequential tables). For 1 < j < n, the information 
specified by BASE[j] and T0P[j] in accord with the conventions stated above 
is moved to new positions specified by NEWBASE [j] , and the values of BASE [j] 
and TOP [ j] are suitably adjusted. This algorithm is based on the easily verified 
fact that the data to be moved downward cannot overlap with any data that is 
to be moved upward, nor with any data that is supposed to stay put. 

Rl. [Initialize.] Set j 4— 1. 

R2. [Find start of shift.] (Now all lists from 1 to j that were to be moved down 
have been shifted into the desired position.) Increase j in steps of 1 until 
finding either 

a) NEWBASE [j] < BASE [j] : Go to R3; or 

b) j > n: Go to R4. 

R3. [Shift list down.] Set 6 <r- BASE [j] - NEWBASE [j] . Set CONTENTS (L - 5) +- 
CONTENTS (L) , for L = BASE [j] + 1, BASE [ j] + 2, . . . , TOP [ j] . (It is possible 
for BASE[j] to equal T0P[j], in which case no action is required.) Set 
BASE [ j] 4- NEWBASE [j] , T0P[j] 4- T0P[j] - 5. Go back to R2. 

R4. [Find start of shift.] (Now all lists from j to n that were to be moved up 
have been shifted into the desired position.) Decrease j in steps of 1 until 
finding either 

a) NEWBASE [ j] > BASE [ j] : Go to R5; or 

b) j = 1: The algorithm terminates. 
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R5. [Shift list up.] Set S t— NEWBASE [ )] - BASE [ j] . Set CONTENTS (L + S) 

CONTENTS (L) , for L = TOP [j] , TOP [j] — 1, . . . , BASE [ j] + 1. (As in step R3, 

no action may actually be needed here.) Set BASE[j] •<— NEWBASE [?'] 

TOP [j] <— TOP [)] + (5. Go back to R4. | 

Notice that stack 1 never needs to be moved. Therefore we should put the 
largest stack first, if we know which one will be largest. 

In Algorithms G and R we have purposely made it possible to have 

0LDT0P [ j] =D[j] = NEWBASE [j + 1] 

foi 1 < j < n; that is, these three tables can share common memory locations 
since their values are never needed at conflicting times. 

We have described these repacking algorithms for stacks, but it is clear that 
they can be adapted to any relatively addressed tables in which the current 
information is contained between BASE [ )] and TOPtj], Other pointers (for 
example, FRONT [j] and REAR [ )] ) could also be attached to the lists, making 
them serve as a queue or deque. See exercise 8, which considers the case of a 
queue in detail. 

The mathematical analysis of dynamic storage-allocation algorithms like 
those above is extremely difficult. Some interesting results appear in the exercises 
below, although they only begin to scratch the surface as far as the general 
behavior is concerned. 

As an example of the theory that can be derived, suppose we consider the 
case when the tables grow only by insertion; deletions and subsequent insertions 
that cancel their effect are ignored. Let us assume further that each table is 
expected to fill at the same rate. This situation can be modeled by imagining 
a sequence of m insertion operations aq, a 2 , . . . , a m , where each a* is an integer 
between 1 and n (representing an insertion on top of stack a,). For example, 
the sequence 1, 1, 2, 2, 1 means two insertions to stack 1, followed by two 
to stack 2, followed by another onto stack 1. We can regard each of the n m 
possible specifications a 1 ,a 2 ,...,a m as equally likely, and then we can ask for 
the average number of times it is necessary to move a word from one location to 
another during the repacking operations as the entire table is built. For the first 
algorithm, starting with all available space given to the nth stack, this question 
is analyzed in exercise 9. We find that the average number of move operations 
required is 



Thus, as we might expect, the number of moves is essentially proportional to the 
square of the number of times the tables grow. The same is true if the individual 
stacks aren t equally likely (see exercise 10). 

The moral of the story seems to be that a very large number of moves will 
be made if a reasonably large number of items is put into the tables. This 
is the price we must pay for the ability to pack a large number of sequential 
tables together tightly. No theory has been developed to analyze the average 
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behavior of Algorithm G, and it is unlikely that any simple model will be able to 
describe the characteristics of real-life tables in such an environment. However, 
exercise 18 provides a worst-case guarantee that the running time will not be 
too bad if the memory doesn’t get too full. 

Experience shows that when memory is only half loaded (that is, when the 
available space equals half the total space) , we need very little rearranging of the 
tables with Algorithm G. The important thing is perhaps that the algorithm 
behaves well in the half-full case and that it at least delivers the right answers 
in the almost-full case. 

But let us think about the almost-full case more carefully. When the tables 
nearly fill memory, Algorithm R takes rather long to perform its job. And to 
make matters worse, OVERFLOW is much more frequent just before the memory 
space is used up. There are very few programs that will come close to filling 
memory without soon thereafter completely overflowing it; and those that do 
overflow memory will probably waste enormous amounts of time in Algorithms G 
and R just before memory is overrun. Unfortunately, undebugged programs will 
frequently overflow memory capacity. To avoid wasting all this time, a possible 
suggestion would be to stop Algorithm G in step G3 if SUM is less than S m j n , 
where the latter is chosen by the programmer to prevent excessive repacking. 
When there are many variable-size sequential tables, we should not expect to 
make use of 100 percent of the memory space before storage is exceeded. 

Further study of Algorithm G has been made by D. S. Wise and D. C. 
Watson, BIT 16 (1976), 442-450. See also A. S. Fraenkel, Inf. Proc. Letters 8 
(1979), 9-10, who suggests working with pairs of stacks that grow towards each 
other. 

EXERCISES 

► 1. {15} In the queue operations given by ( 6 a) and ("a), how many items can be in 
the queue at one time without OVERFLOW occurring? 

► 2. [22] Generalize the method of ( 6 a) and ( 7 a) so that it will apply to any deque with 
fewer than M elements. In other words, give specifications for the other two operations, 
“delete from rear” and “insert at front.” 

3. [21] Suppose that MIX is extended as follows: The I-held of each instruction is to 
have the form 8 I 1 + 1 2 , where 0 < Ii < 8 , 0 < I 2 < 8 . In assembly language one writes 
“OP ADDRESS.Ii :I 2 ” or (as presently) “OP ADDRESS, I 2 ” if Ii = 0. The meaning is to 
perform first the “address modification” Ii on ADDRESS, then to perform the “address 
modification” I 2 on the resulting address, and finally to perform the OP with the new 
address. The address modifications are defined as follows: 

0: M = A 

1: M = A + rll 

2: M = A + rI2 

6 : M = A + rI 6 

7: M = resulting address defined from the “ADDRESS , Ii : I 2 ” fields found in loca- 
tion A. The case Ii = I 2 = 7 in location A is not allowed. (The reason 
for the latter restriction is discussed in exercise 5.) 
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Here A denotes the address before the operation, and M denotes the resulting address 
after the address modification. In all cases the result is undefined if the value of M 
does not fit in two bytes and a sign. The execution time is increased by one unit for 
each indirect-addressing” (modification 7) operation performed. 

As a nontrivial example, suppose that location 1000 contains “NOP 1000,1:7": 
location 1001 contains “NOP 1000,2”; and index registers 1 and 2 respectively contain 
1 and 2. Then the command “LDA 1000,7:2” is equivalent to “LDA 1004”. because 

1000,7:2 = (1000, 1:7), 2 = (1001, 7), 2 = (1000, 2), 2 = 1002,2 = 1004. 

a) Using this indirect addressing feature (if necessary), show how to simplify the 
coding on the right-hand side of (8) so that two instructions are saved per reference to 
the table. How much faster is your code than (8)? 

b) Suppose there are several tables whose base addresses are stored in locations 
BASE + 1, BASE + 2, BASE + 3, . . . ; how can the indirect addressing feature be used to 
bring the Ith element of the Jth table into register A in one instruction, assuming that 
I is in rll and J is in rI2? 

c) What is the effect of the instruction “ENT4 X,7”, assuming that the (3:3)-field 
in location X is zero? 

4. [ 25 ] Assume that MIX has been extended as in exercise 3. Show how to give a 
single instruction (plus auxiliary constants) for each of the following actions: 

a) To loop indefinitely because indirect addressing never terminates. 

b) To bring into register A the value LINK (LINK (x ) ) , where the value of link variable 
x is stored in the (0:2) field of the location whose symbolic address is X. the value 
of LINK (a;) is stored in the (0:2) field of location x, etc., assuming that the (3:3) 
fields in these locations are zero. 

c) To bring into register A the value LINK (LINK (LINK Or) )) , under assumptions like 
those in (b). 

d) To bring into register A the contents of location rll + rI2 + rI3 + rI4 + r!5 + rI6. 

e) To quadruple the current value of rI6. 

► 5. [3j] The extension of MIX suggested in exercise 3 has an unfortunate restriction 
that “7 : 7” is not allowed in an indirectly addressed location. 

a) Give an example to indicate that, without this restriction, it would probably be 
necessary for the MIX hardware to be capable of maintaining a long internal stack 
of three-bit items. (This would be prohibitively expensive hardware, even for a 
mythical computer like MIX.) 

b) Explain why such a stack is not needed under the present restriction; in other 
words, design an algorithm with which the hardware of a computer could perform 
the desired address modifications without much additional register capacity. 

c) Give a milder restriction than that of exercise 3 on the use of 7: 7 that alleviates the 
difficulties of exercise 4(c), yet can be cheaply implemented in computer hardware. 

6. [10] Starting with the memory configuration shown in Pig. 4, determine which of 
the following sequences of operations causes overflow or underflow: 

( a ) III (b) I 2 ; (c) I 3 ; (d) I4I4I4I4I4; (e) D 2 D 2 I 2 l2l2. 

7. [ 12 ] Step G4 of Algorithm G indicates a division by the quantity INC. Can INC 
ever be zero at that point in the algorithm? 

► 8. [ 26 ] Explain how to modify (9), (10), and the repacking algorithms for the case 
that one or more of the lists is a queue being handled circularly as in (6a) and (7a). 
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► 9. [M27] Using the mathematical model described near the end of the text, prove 

that Eq. (14) is the expected number of moves. (Note that the sequence 1, 1, 4, 2, 3, 

I. 2, 4, 2, 1 specifies 0 + 0 + 0 + 1 + 14-3 + 2 + 0 + 3 + 6 = 16 moves.) 

10. [ M28 ] Modify the mathematical model of exercise 9 so that some tables are 
expected to be larger than others: Let p k be the probability that a,j = A, for 1 < j < m, 
1 < A < n. Thus pi + p 2 + • • • +p n = 1; the previous exercise considered the special case 
p k = 1 jn for all A. Determine the expected number of moves, as in Eq. (14), for this 
more general case. It is possible to rearrange the relative order of the n lists so that 
the lists expected to be longer are put to the right (or to the left) of the lists that are 
expected to be shorter; what relative order for the n lists will minimize the expected 
number of moves, based on pi, p 2 , . . . , p„? 

II. [M30] Generalize the argument of exercise 9 so that the first t insertions in any 
stack cause no movement, while subsequent insertions are unaffected. Thus if t = 2, 
the sequence in exercise 9 specifies 0 + 0 + 0 + 0 + 0 + 3 + 0 + 0 + 3 + 6= 12 moves. What 
is the average total number of moves under this assumption? [This is an approximation 
to the behavior of the algorithm when each stack starts with t available spaces.] 

12. [M28] The advantage of having two tables coexist in memory by growing towards 
each other, rather than by having them kept in separate independently bounded areas, 
may be quantitatively estimated (to a certain extent) as follows. Use the model of 
exercise 9 with n = 2; for each of the 2 m equally probable sequences ai, a 2 5 ♦ * • 1 &m •> let 
there be Aq Is and A 2 2s. (Here Aq and A 2 are the respective sizes of the two tables 
after the memory is full. We are able to run the algorithm with m = Aq + A 2 locations 
when the tables are adjacent, instead of 2max(Aq,A 2 ) locations to get the same effect 
with separate tables.) 

What is the average value of max(Aq, A 2 )? 

13. [HM 42 ] The value max(Aq,A 2 ) investigated in exercise 12 will be even greater if 
larger fluctuations in the tables are introduced by allowing random deletions as well as 
random insertions. Suppose we alter the model so that with probability p the sequence 
value cij is interpreted as a deletion instead of an insertion; the process continues until 
Aq +fc 2 (the total number of table locations in use) equals m. A deletion from an empty 
list causes no effect. 

For example, if m = 4 it can be shown that we get the following probability 
distribution when the process stops: 


(At, A 2 ) — 

(0,4) 

(1.3) 

(2,2) 

(3,1) 

(4,0) 

with probability 

1 

1 

6 — 6p + 2p 2 

1 

1 

16 — 12p + 4p 2 ’ 

4’ 

16 — 12p + 4p 2 ’ 

4’ 

16 — 12p + 4p 2 

Thus as p increases. 

, the difference 

between 

Aq and A 2 tends to 

increase. It is not 


difficult to show that in the limit as p approaches unity, the distribution of Aq becomes 
essentially uniform, and the limiting expected value of max(Aq,A 2 ) is exactly |m + 
[rn odd]. This behavior is quite different from that in the previous exercise (when 
p = 0); however, it may not be extremely significant, since when p approaches unity, 
the amount of time taken to terminate the process rapidly approaches infinity. The 
problem posed in this exercise is to examine the dependence of max(Ai,fc 2 ) on p and 
rn. and to determine asymptotic formulas for fixed p (like p = | ) as m approaches 
infinity. The case p = A is particularly interesting. 
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14. [IIM43] Generalize the result of exercise 12 to arbitrary n > 2, by showing that, 
when n is fixed and m approaches infinity, the quantity 

max(fci, fc 2 , . . . , fc„ ) 
k\ \k 2 \ . . . k n ! 

k± ,k2,...,k n >0 

has the asymptotic form m/n+c ny /m+0(l). Determine the constants c 2 , C 3 , C 4 , and C 5 . 

15. [40] Using a Monte Carlo method, simulate the behavior of Algorithm G under 
varying distributions of insertions and deletions. What do your experiments imply 
about the efficiency of Algorithm G? Compare its performance with the algorithm 
given earlier that shifts up and down one node at a time. 

16. [20] The text illustrates how two stacks can be located so they grow towards each 
other, thereby making efficient use of a common memory area. Can two queues, or a 
stack and a queue, make use of a common memory area with the same efficiency? 

17. [30] If a is any sequence of insertions and deletions such as ( 12 ), let s 0 (cr) be the 
number of stack overflows that occur when the simple method of Fig. 4 is applied to cr 
with initial conditions ( 11 ), and let si(<r) be the corresponding number of overflows 
with respect to other initial conditions such as ( 13 ). Prove that s 0 (cr) < s 1 ( ( t)+L so -L 0 . 

► 18. [ M30 ] Show that the total running time for any sequence of m insertions and/or 
deletions by Algorithms G and R is 0(m+n x a k /(l-a k )), where a k is the fraction 
of memory occupied on the most recent repacking previous to the fcth operation; a k = 0 
before the first repacking. (Therefore if the memory never gets more than, say, 90% 
full, each operation takes at most 0(n) units of time in an amortized sense, regardless 
of the total memory size.) Assume that Loo - L 0 > n 2 . 

► 19. [16] ( 0-origin indexing .) Experienced programmers learn that it is generally wise 
to denote the elements of a linear list by X[0] , X [1] , . . . , X [n - 1] , instead of using the 
more traditional notation X [1] , X [2] , . . . , X [n] . Then, for example, the base address 
L 0 m ( 1 ) points to the smallest cell of the array. 

Revise the insertion and deletion methods ( 2 a), ( 3 a), ( 6 a), and ( 7 a) for stacks and 
queues so that they conform to this convention. In other words, change them so that 
the list elements will appear in the array X[0], X [1] , ..., X[M- 1] , instead of X[ll 
X [ 2 ] , . . . , X [H] . 

2.2.3. Linked Allocation 

Instead of keeping a linear list in sequential memory locations, we can make use 
of a much more flexible scheme in which each node contains a link to the next 
node of the list. 


m! 

n m 


E 

^l+^H b 


Sequential allocation: Linked allocation: 


Address 

Contents 

Address 

Contents 

L 0 + c: 

Item 1 

A: 

Item 1 

B 

Lq + 2c: 

Item 2 

B: 

Item 2 

C 

Lq T 3c: 

Item 3 

C: 

Item 3 

D 

L 0 + 4c: 

Item 4 

D: 

Item 4 

E 

L 0 + 5c: 

Item 5 

E: 

Item 5 

A 
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Here A, B, C, D, and E are arbitrary locations in the memory, and A is the null 
link (see Section 2.1). The program that uses this table in the case of sequential 
allocation would have an additional variable or constant whose value indicates 
that the table is five items in length, or else this information would be specified 
by a sentinel code within item 5 or in the following location. A program for 
linked allocation would have a link variable or link constant that points to A; all 
the other items of the list can be found from address A. 

Recall from Section 2.1 that links are often shown simply by arrows, since 
the actual memory locations occupied are usually irrelevant. The linked table 
above might therefore be shown as follows: 


FIRST 



(1) 

Here FIRST is a link variable pointing to the first node of the list. 

We can make several obvious comparisons between these two basic forms of 
storage: 

1) Linked allocation takes up additional memory space for the links. This 
can be the dominating factor in some situations. However, we frequently find 
that the information in a node does not take up a whole word anyway, so there 
is already space for a link field present. Also, it is possible in many applications 
to combine several items into one node so that there is only one link for several 
items of information (see exercise 2.5-2). But even more importantly, there is 
often an implicit gain in storage by the linked memory approach, since tables 
can overlap, sharing common parts; and in many cases, sequential allocation will 
not be as efficient as linked allocation unless a rather large number of additional 
memory locations are left vacant anyway. For example, the discussion at the end 
of the previous section explains why the systems described there are necessarily 
inefficient when memory is densely loaded. 

2) It is easy to delete an item from within a linked list. For example, to 
delete item 3 we need only change the link associated with item 2. But with 
sequential allocation such a deletion generally implies moving a large part of the 
list up into different locations. 

3) It is easy to insert an item into the midst of a list when the linked scheme 
is being used. For example, to insert an item 2| into (l) we need to change only 
two links: 



By comparison, this operation would be extremely time-consuming in a long 
sequential table. 

4) References to random parts of the list are much faster in the sequential 
case. To gain access to the fcth item in the list, when k is a variable, takes a fixed 
time in the sequential case, but we need k iterations to march down to the right 
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place in the linked case. Thus the usefulness of linked memory is predicated on 
the fact that in the large majority of applications we want to walk through lists 
sequentially, not randomly; if items in the middle or at the bottom of the list 
are needed, we try to keep an additional link variable or list of link variables 
pointing to the proper places. 

5) The linked scheme makes it easier to join two lists together, or to break 
one apart into two that will grow independently. 

6 ) The linked scheme lends itself immediately to more intricate structures 
than simple linear lists. We can have a variable number of variable-size lists; 
any node of the list may be a starting point for another list; the nodes may 
simultaneously be linked together in several orders corresponding to different 
lists; and so on. 

7) Simple operations, like proceeding sequentially through a list, are slightly 
faster for sequential lists on many computers. For MIX, the comparison is between 
“INCl c” and “LD1 0,1 (LINK)”, which is only one cycle different, but many 
machines do not enjoy the property of being able to load an index register from 
an indexed location. If the elements of a linked list belong to different pages in 
a bulk memory, the memory accesses might take significantly longer. 

Thus we see that the linking technique, which frees us from any constraints 
imposed by the consecutive nature of computer memory, gives us a good deal 
more efficiency in some operations, while we lose some capabilities in other cases. 
It is usually clear which allocation technique will be most appropriate in a given 
situation, and both methods are often used in different lists of the same program. 

In the next few examples we will assume for convenience that a node has 
one word and that it is broken into the two fields INFO and LINK: 


INFO 


LINK 


(3) 


The use of linked allocation generally implies the existence of some mecha- 
nism for finding empty space available for a new node, when we wish to insert 
some newly created information onto a list. This is usually done by having a 
special list called the list of available space. We will call it the AVAIL list (or the 
AVAIL stack, since it is usually treated in a last-in-first-out manner). The set of 
all nodes not currently in use is linked together in a list just like any other list; 
the link variable AVAIL refers to the top element of this list. Thus, if we want 
to set link variable X to the address of a new node, and to reserve that node for 
future use, we can proceed as follows: 


X <— AVAIL, AVAIL <— LINK (AVAIL). ( 4 ) 

This effectively removes the top of the AVAIL stack and makes X point to the 
node just removed. Operation ( 4 ) occurs so often that we have a special notation 
for it: “X -t= AVAIL” will mean X is set to point to a new node. 

When a node is deleted and no longer needed, process ( 4 ) can be reversed: 

LINK (X) AVAIL, AVAIL X. ( 5 ) 
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This operation puts the node addressed by X back onto the list of raw material; 
we denote ( 5 ) by “AVAIL <t= X”. 

Several important things have been omitted from this discussion of the AVAIL 
stack. We did not say how to set it up at the beginning of a program; clearly 
this can be done by (a) linking together all nodes that are to be used for linked 
memory, (b) setting AVAIL to the address of the first of these nodes, and (c) 
making the last node link to A. The set of all nodes that can be allocated is 
called the storage pool. 

A more important omission in our discussion was the test for overflow: We 
neglected to check in ( 4 ) if all available memory space has been taken. The 
operation X 4= AVAIL should really be defined as follows: 

if AVAIL = A, then OVERFLOW; 

otherwise X 4- AVAIL. AVAIL 4- LINK (AVAIL) . (6) 

The possibility of overflow must always be considered. Here OVERFLOW generally 
means that we terminate the program with regrets; or else we can go into a 
“garbage collection” routine that attempts to find more available space. Garbage 
collection is discussed in Section 2.3.5. 

There is another important technique for handling the AVAIL stack: We 
often do not know in advance how much memory space should be used for the 
storage pool. There may be a sequential table of variable size that wants to 
coexist in memory with the linked tables; in such a case we do not want the 
linked memory area to take any more space than is absolutely necessary. So 
suppose that we wish to place the linked memory area in ascending locations 
beginning with L 0 and that this area is never to extend past the value of variable 
SEQMIN (which represents the current lower bound of the sequential table). Then 
we can proceed as follows, using a new variable POOLMAX: 

a) Initially set AVAIL 4— A and POOLMAX 4— Lo. 

b) The operation X -£= AVAIL becomes the following: 

'■If AVAIL ^ A, then X 4- AVAIL, AVAIL 4 - LINK (AVAIL). 

Otherwise set X 4— POOLMAX and POOLMAX 4— X + c, where c is the ( 7 ) 

node size; OVERFLOW now occurs if POOLMAX > SEQMIN.” 

c) When other parts of the program attempt to decrease the value of SEQMIN, 

they should sound the OVERFLOW alarm if SEQMIN < POOLMAX. 

d) The operation AVAIL •$= X is unchanged from ( 5 ). 

This idea actually represents little more than the previous method with a special 
recovery procedure substituted for the OVERFLOW situation in (6). The net 
effect is to keep the storage pool as small as possible. Many people like to 
use this idea even when all lists occupy the storage pool area (so that SEQMIN is 
constant), since it avoids the rather time-consuming operation of initially linking 
all available cells together and it facilitates debugging. We could, of course, put 
the sequential list on the bottom and the pool on the top, having POOLMIN and 
SEQMAX instead of POOLMAX and SEQMIN. 
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Thus it is quite easy to maintain a pool of available 
nodes, in such a way that free nodes can efficiently be 
found and later returned. These methods give us a source 
of raw material to use in linked tables. Our discussion 
was predicated on the implicit assumption that all nodes 
have a fixed size, c; the cases that arise when different 
sizes of nodes are present are very important, but we 
will defer that discussion until Section 2.5. Now we will 
consider a few of the most common list operations in the 
special case where stacks and queues are involved. 

The simplest kind of linked list is a stack. Figure 5 
shows a typical stack, with a pointer T to the top of the 
stack. When the stack is empty, this pointer will have 
the value A. Fig. 5. A linked stack. 

It is clear how to insert (“push down”) new information Y onto the top of 
such a stack, using an auxiliary pointer variable P. 



P <t= AVAIL, INFO(P) 4— Y, LINK(P) 4— T, Tf-P. (8) 

Conversely, to set Y equal to the information at the top of the stack and to “pop 
up” the stack: 


If T = A, then UNDERFLOW; 

otherwise set P 4- T, T <— LINK(P), Y 4- INFO(P), AVAIL -1= P. ( 9 ) 

These operations should be compared with the analogous mechanisms for se- 
quentially allocated stacks, (2a) and ( 3 a) in Section 2.2.2. The reader should 
study (8) and (9) carefully, since they are extremely important operations. 

Before looking at the case of queues, let us see how the stack operations can 
be^expressed conveniently in programs for MIX. A program for insertion, with 
P = rll, can be written as follows: 


INFO EQU 

0:3 

Definition of the INFO field 

LINK EQU 

4:5 

Definition of the LINK field 

LD1 

AVAIL 

P 4- AVAIL. 


J1Z 

OVERFLOW 

Is AVAIL = A? 


LDA 

0,1 (LINK) 

I 

> P <= AVAIL 

STA 

AVAIL 

AVAIL 4- LINK(P) . J 


LDA 

Y 



STA 

0,1 (INFO) 

INFO (P) 4- Y. 


LDA 

T 



STA 

0,1 (LINK) 

LINK(P) 4- T. 


ST1 

T 

T 4- P. 

1 


(10) 


This takes 17 units of time, compared to 12 units for the comparable operation 
with a sequential table (although OVERFLOW in the sequential case would in many 
cases take considerably longer). In this program, as in others to follow in this 
chapter, OVERFLOW denotes either an ending routine or a subroutine that finds 
more space and returns to location rJ - 2. 
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A program for deletion is equally simple: 


LD1 

T 

P <— T. 


J1Z 

LDA 

UNDERFLOW 
0,1 (LINK) 

Is T = A? 


STA 

LDA 

T 

0,1 (INFO) 

T +- LINK(P) . 


STA 

LDA 

Y 

AVAIL 

Y <- INFO (P) . 

1 


STA 

0,1 (LINK) 

LINK(P) <- AVAIL. 

l AVAIL -<= P 

ST1 

AVAIL 

AVAIL <- P. J 

■ 1 


It is interesting to observe that each of these operations involves a cyclic 
permutation of three links. For example, in the insertion operation let P be the 
value of AVAIL before the insertion; if P A, we find that after the operation 


the value of AVAIL has become the previous value of LINK(P), 
the value of LINK(P) has become the previous value of T, and 
the value of T has become the previous value of AVAIL. 


So the insertion process (except for setting INFO(P) 
tlon AVAIL 


Y) is the cyclic permuta- 


LINK(P) 


Similarly in the case of deletion, where P has the value of T before the operation 
and we assume that P 7^ A, we have Y t— INFO(P) and 


AVAIL 



LINK(P) 


The fact that the permutation is cyclic is not really a relevant issue, since any 
permutation of three elements that moves every element is cyclic. The important 
point is rather that precisely three links are permuted in these operations. 

The insertion and deletion algorithms of (8) and (9) have been described 
for stacks, but they apply much more generally to insertion and deletion in any 
linear list. Insertion, for example, is performed just before the node pointed to 
by link variable T. The insertion of item 2^ in (2) above would be done by using 
operation (8) with T = LINK (LINK (FIRST) ) . 

Linked allocation applies in a particularly convenient way to queues. In this 
case it is easy to see that the links should run from the front of the queue towards 
the rear, so that when a node is removed from the front, the new front node is 
directly specified. We will make use of pointers F and R, to the front and rear: 


i 


Except for R, this diagram is abstractly identical to Fig. 5 on page 258. 


(12) 
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Whenever the layout of a list is designed, it is important to specify all 
conditions carefully, particularly for the case when the list is empty. One of 
the most common programming errors connected with linked allocation is the 
failure to handle empty lists properly; the other common error is to forget 
about changing some of the links when a structure is being manipulated. In 
order to avoid the first type of error, we should always examine the “boundary 
conditions” carefully. To avoid making the second type of error, it is helpful to 
draw “before and after” diagrams and to compare them, in order to see which 
links must change. 

Let’s illustrate the remarks of the preceding paragraph by applying them to 
the case of queues. First consider the insertion operation: If (12) is the situation 
before insertion, the picture after insertion at the rear of the queue should be 



(The notation used here implies that a new node has been obtained from the 
AVAIL list.) Comparing (12) and (13) shows us how to proceed when inserting 
the information Y at the rear of the queue: 


P 4= AVAIL, INFO(P) t— Y, LINK(P) A, LINK(R) t— P, R <- p. ( 14 ) 


Let us now consider the boundary situation when the queue is empty: In 
this case the situation before insertion is yet to be determined, and the situation 
“after” is 


F 


it 

AVAIL 


i 


K R 


(15) 


It is desirable to have operations (14) apply in this case also, even if insertion 
into an empty queue means that we must change both F and R, not only R. 
We find that (14) will work properly if R = LOC(F) when the queue is empty, 
assuming that F = LINK(L0C(F)) ; the value of variable F must be stored in the 
LINK field of its location if this idea is to work. I11 order to make the testing for 
an empty queue as efficient as possible, we will let F = A in this case. Our policy 
is therefore that 


an empty queue is represented by F = A and R = LOC(F) . 

If the operations (14) are applied under these circumstances, we obtain (15). 

The deletion operation for queues is derived in a similar fashion. If (12) is 
the situation before deletion, the situation afterwards is 



AVAIL 


For the boundary conditions we must make sure that the deletion operation 
works when the queue is empty either before or after the operation. These 









2.2.3 


LINKED ALLOCATION 261 


considerations lead us to the following way to do queue deletion in general: 

If F = A, then UNDERFLOW; 

otherwise set P <— F, F <- LINK(P), Y <- INFO(P), AVAIL 4 = P, (17) 

and if F — A, then set R e- LOC(F) . 

Notice that R must be changed when the queue becomes empty; this is precisely 
the type of “boundary condition” we should always be watching for. 

These suggestions are not the only way to represent queues in a linearly 
linked fashion; exercise 30 describes a somewhat more natural alternative, and 
we will give other methods later in this chapter. Indeed, none of the operations 
above are meant to be prescribed as the only way to do something; they are 
intended as examples of the basic means of operating with linked lists. The 
reader who has had only a little previous experience with such techniques will 
find it helpful to reread the present section up to this point before going on. 

So far in this chapter we have discussed how to perform certain operations 
on tables, but our discussions have always been “abstract,” in the sense that 
we never exhibited actual programs in which the particular techniques were 
useful. People aren’t generally motivated to study abstractions of a problem until 
they’ve seen enough special instances of the problem to arouse their interest. The 
operations discussed so far — manipulations of variable-size lists of information 
by insertion and deletion, and the use of tables as stacks or queues — are of such 
wide application, it is hoped that the reader will have encountered them often 
enough already to grant their importance. But now we will leave the realm of 
the abstract as we begin to study a series of significant practical examples of the 
techniques of this chapter. 

Our first example is a problem called topological sorting , which is an impor- 
tant process needed in connection with network problems, with so-called PERT 
charts, and even with linguistics; in fact, it is of potential use whenever we have 
a problem involving a partial ordering. A partial ordering of a set 5 is a relation 
between the objects of 5, which we may denote by the symbol ‘H”, satisfying 
the following properties for any objects x, y, and z (not necessarily distinct) 
in S: 

i) If x ■< y and y < z, then x < z. (Transitivity.) 

ii) If x < y and y < x, then x — y. (Antisymmetry.) 

iii) x < x. (Reflexivity.) 

The notation x A y may be read “x precedes or equals y" If x <y and i/j, 
we write x < y and say “x precedes y.” It is easy to see from (i), (ii), and (iii) 
that we always have 

i') If x < y and y -< z, then x -< z. (Transitivity.) 
ii') If x -< y, then y 7 ^ x. (Asymmetry.) 
iii') 1 / 1 . (Irreflexivity.) 
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The relation denoted by y / x means “y does not precede x.” If we start with 
a relation -< satisfying properties (i'), (ii'), and (in'), we can reverse the process 
above and define x < y ii x ^ y or x = y, then properties (i), (ii), and (iii) are 
true. Therefore we may regard either properties (i), (ii), (iii) or properties (i'), 
(ii'), (iii') as the definition of partial order. Notice that property (ii') is actually 
a consequence of (i') and (iii'), although (ii) does not follow from (i) and (iii). 

Partial orderings occur quite frequently in everyday life as well as in math- 
ematics. As examples from mathematics we can mention the relation x < y 
between real numbers x and y, the relation x C y between sets of objects; 
the relation x\y (x divides y) between positive integers. In the case of PERT 
networks, S is a set of jobs that must be done, and the relation u x -< y" means 
“ x must be done before y." 


Fig. 6. A partial ordering. 



We will naturally assume that 5 is a finite set, since we want to work with S 
inside a computer. A partial ordering on a finite set can always be illustrated 
by drawing a diagram such as Fig. 6, in which the objects are represented by 
small boxes and the relation is represented by arrows between these boxes; x -< y 
means there is a path from the box labeled x to box y that follows the direction 
of the arrows. Property (ii) of partial ordering means there are no closed loops 
(no paths that close on themselves) in the diagram. If an arrow were drawn from 
4 to 1 in Fig. 6, we would no longer have a partial ordering. 

The problem of topological sorting is to embed the partial order in a linear 
order-, that is, to arrange the objects into a linear sequence a\a 2 . . ,a n such that 
whenever aj -< a k , we have j < k. Graphically, this means that the boxes are to 
be rearranged into a line so that all arrows go towards the right (see Fig. 7). It 
is not immediately obvious that such a rearrangement is possible in every case, 
although such a rearrangement certainly could not be done if any loops were 
present. Therefore the algorithm we will give is interesting not only because it 
does a useful operation, but also because it proves that this operation is possible 
for every partial ordering. 

As an example of topological sorting, imagine a large glossary containing 
definitions of technical terms. We can write w 2 Wi if the definition of word w\ 
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Fig. 7. The ordering relation of Fig. 6 after topological sorting. 

depends directly or indirectly on that of word u> 2 . This relation is a partial 
ordering provided that there are no “circular” definitions. The problem of 
topological sorting in this case is to find a way to arrange the words in the 
glossary so that no term is used before it has been defined. Analogous problems 
arise in writing programs to process the declarations in certain assembly and 
compiler languages; they also arise in writing a user’s manual describing a 
computer language or in writing textbooks about information structures. 

There is a very simple way to do topological sorting: We start by taking an 
object that is not preceded by any other object in the ordering. This object may 
be placed first in the output. Now we remove this object from the set S; the 
resulting set is again partially ordered, and the process can be repeated until the 
whole set has been sorted. For example, in Fig. 6 we could start by removing 1 
or 9; after 1 has been removed, 3 can be taken, and so on. The only way in which 
this algorithm could fail would be if there were a nonempty partially ordered set 
in which every element was preceded by another; for in such a case the algorithm 
would find nothing to do. But if every element is preceded by another, we could 
construct an arbitrarily long sequence 61 , 62 , 63 , ■ • • * n which 6 J+ i -< bj. Since S 
is finite, we must have b 3 = 6*, for some j < k\ but j < k implies that 6^ A bj + 1 , 
hence bj = 6 ^ contradicts (ii). 

In order to implement this process efficiently by computer, we need to 
be ready to perform the actions described above, namely to locate objects 
that are not preceded by any others, and to remove them from the set. Our 
implementation is also influenced by the desired input and output characteristics. 
The most general program would accept alphabetic names for the objects and 
would allow gigantic sets of objects to be sorted — more than could possibly fit 
in the computer memory at once. Such complications would obscure the main 
points we are trying to make here, however; the handling of alphabetic data can 
be done efficiently by using the methods of Chapter 6 , and the handling of large 
networks is left as an interesting project for the reader. 

Therefore we will assume that the objects to be sorted are numbered from 1 
to n in any order. The input of the program will be on tape unit 1: Each tape 
record contains 50 pairs of numbers, where the pair ( j , k) means that object j 
precedes object k. The first pair, however, is (0, n), where n is the number of 
objects. The pair (0,0) terminates the input. We shall assume that n plus the 
number of relation pairs will fit comfortably in memory; and we shall assume 
that it is not necessary to check the input for validity. The output is to be the 
numbers of the objects in sorted order, followed by the number 0 , on tape unit 2 . 
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As an example of the input, we might have the relations 


9^2, 3^7, 7 -< 5, 5 -< 8 , 8 -< 6 , 4^6, 1^3, 7^4, 9^5, 2^8. (18) 

It is not necessary to give any more relations than are needed to characterize 
the desired partial ordering. Thus, additional relations like 9^8 (which can 
be deduced from 9 5 and 5^8) may be omitted from or added to the input 

without harm. In general, it is necessary to give only the relations corresponding 
to arrows on a diagram such as Fig. 6. 

The algorithm that follows uses a sequential table X [1] , X [2] x[n], 

and each node X [Ad has the form 


+ 

0 

COUNT [A;] 

TOP [Ad 


Here COUNT [Ad is the number of direct predecessors of object k (the number of 
relations j k that have appeared in the input), and TOP [A;] is a link to the 
beginning of the list of direct successors of object k. The latter list contains 
entries in the format 


+ 

0 

SUC 

1 

NEXT 


where SUC is a direct successor of k and NEXT is the next item of the list. As an 
example of these conventions, Fig. 8 shows the schematic contents of memory 
corresponding to the input (18). 


k 

count [Ad 
top [A-] 


sue 

NEXT 


SUC 

NEXT 



Fig. 8. Computer representation of Fig. 6 corresponding to the relations (18). 


Using this memory layout, it is not difficult to work out the algorithm. 
We want to output the nodes whose COUNT field is zero, then to decrease the 
COUNT fields of all successors of those nodes by one. The trick is to avoid doing 
any searching for nodes whose COUNT field is zero, and this can be done by 
maintaining a queue containing those nodes. The links for this queue are kept 
m the COUNT field, which by now has served its previous purpose; for clarity in 
the algorithm below, we use the notation QLINK [AO to stand for COUNT [Ad when 
that field is no longer being used to keep a count. 
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Algorithm T (Topological sort). This algorithm inputs a sequence of relations 
j -< k , indicating that object j precedes object k in a certain partial ordering, 
assuming that 1 < j, k < n. The output is the set of n objects embed- 
ded in a linear order. The internal tables used are: QLINK[0], COUNT [1] = 
QLINK [1] , COUNT [2] = QLINK [2] , . . . , COUNT [n] = QLINK [n] ; TOP [1] , TOP [2] , 

. . . , TOP [n] ; a storage pool with one node for each input relation and with SUC 
and NEXT fields as shown above; P, a link variable used to refer to the nodes in 
the storage pool; F and R, integer-valued variables used to refer to the front and 
rear of a queue whose links are in the QLINK table; and N, a variable that counts 
how many objects have yet to be output. 

Tl. [Initialize.] Input the value of n. Set COUNT [fc] 4- 0 and T0P[fc] 4- A for 
1 < k < n. Set N 4— n. 

T2. [Next relation.] Get the next relation “j -< k" from the input; if the input 
has been exhausted, however, go to T4. 

T3. [Record the relation.] Increase COUNT [fc] by one. Set 

P 4= AVAIL, SUC(P) 4- k, NEXT(P) 4-T0P[j], TOP [ j] 4- P. 

(This is operation (8).) Go back to T2. 

T4. [Scan for zeros.] (At this point we have completed the input phase; the input 
(18) would now have been transformed into the computer representation 
shown in Fig. 8. The next job is to initialize the queue of output, which 
is linked together in the QLINK field.) Set R 4— 0 and QLINK [0] 4— 0. For 
1 < k < n examine COUNT [fc] , and if it is zero, set QLINK [R] 4— k and 
R 4— k. After this has been done for all k, set F 4— QLINK [0] (which will 
contain the first value k encountered for which COUNT [fc] was zero) . 

T5. [Output front of queue.] Output the value of F. If F = 0, go to T8; otherwise, 
set N 4— N — 1, and set P 4- T0P[F]. (Since the QLINK and COUNT tables 
overlap, we have QLINK [R] = 0; therefore the condition F = 0 occurs when 
the queue is empty.) 

T6. [Erase relations.] If P = A, go to T7. Otherwise decrease COUNT [SUC (P)] 
by one, and if it has thereby gone down to zero, set QLINK [R] 4— SUC(P) 
and R 4— SUC(P). Set P 4— NEXT(P) and repeat this step. (We are removing 
all relations of the form “F -< k ” for some k from the system, and putting 
new nodes into the queue when all their predecessors have been output.) 

T7. [Remove from queue.] Set F 4— QLINK [F] and go back to T5. 

T8. [End of process.] The algorithm terminates. If N — 0, we have output all of 
the object numbers in the desired “topological order,” followed by a zero. 
Otherwise the N object numbers not yet output contain a loop, in violation 
of the hypothesis of partial order. (See exercise 23 for an algorithm that 
prints out the contents of one such loop.) | 

The reader will find it helpful to try this algorithm by hand on the input 
(18). Algorithm T shows a nice interplay between sequential memory and linked 
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memory techniques. Sequential memory is used for the main table X [1] 

XW. which c °ntains the COUNT [A,-] and TOP [A] entries, because we want to 
make references to “random” parts of this table in step T3. (If the input were 
alphabetic, however, another type of table would be used for speedier search, as 
m Chapter 6 .) Linked memory is used for the tables of “immediate successors,” 
since those table entries have no particular order in the input. The queue of 
nodes waiting to be output is kept in the midst of the sequential table by linking 
the nodes together in output order. This linking is done by table index instead 
of by address; m other words, when the front of the queue is X [A] , we have 
F — A instead of F = L0C(X[A]). The queue operations used in steps T4, T 6 . 
and T7 are not identical to those in ( 14 ) and ( 17 ), since we are taking advantage 
of special properties of the queue in this system; no nodes need to be created or 
returned to available space during this part of the algorithm. 

The coding of Algorithm T in MIX assembly language has a few additional 
points of interest. Since no deletion from tables is made in the algorithm (because 
no storage must be freed for later use), the operation P 4 = AVAIL can be done 
m an extremely simple way, as shown in lines 19 and 32 below; we need not 
keep any linked pool of memory, and we can choose new nodes consecutively. 

le program includes complete input and output with magnetic tape, according 
to the conventions mentioned above, but buffering is omitted for the sake of 
simplicity. The reader should not find it very difficult to follow the details of 
the coding in this program, since it corresponds directly to Algorithm T but 
with slight changes for efficiency. The efficient use of index registers, which is an 
important aspect of linked memory processing, is illustrated here. 

Program T ( Topological sort). In this program, the following equivalences 
should be noted: rI 6 = N, rI5 = buffer pointer, rI4 = k , rI3 = j and R. rI2 = 
AVAIL and P, rll = F, TOP [ j] =X+j( 4:5), COUNT [A] = QLINK [A] = X + A(2:3). 

01 * BUFFER AREA AND FIELD DEFINITIONS 

02 COUNT EQU 2:3 

03 QLINK EQU 2:3 

04 TOP EQU 4:5 

05 SUC EQU 2:3 

06 NEXT EQU 4:5 

07 TAPEIN EQU 1 

08 TAPEOUT EQU 2 

09 BUFFER ORIG *+100 

10 CON -1 

11 * INPUT PHASE 

12 TOPSORT IN BUFFER (TAPEIN) 1 

13 JBUS * (TAPEIN) 

14 1H LD6 BUFFER+1 1 

15 ENT4 0,6 1 

16 STZ X ,4 ri + 1 

17 DEC4 1 n+ i 

18 J4NN *-2 n + 1 

19 ENT2 X,6 1 


Definition of symbolic 
names of fields 


Input is on tape unit 1 
Output is on tape unit 2 
Tape buffer area 
Sentinel at end of buffer 

Tl. Initialize. Read in the first 

tape block; wait for completion. 

N <— n. 

Set COUNT [A] +- 0 and TOP [A] «- A, 
for 0 < A < n. 

(Anticipate QLINK [0] +- 0 in step T4.) 
Available storage starts after X [n] . 
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Fig. 9. Topological sorting. 


20 


ENT5 

BUFFER+2 

1 

Prepare to read the first pair ( j , k). 

21 

2H 

LD3 

0,5 

m + b 

T2. Next relation. 

22 


J3P 

3F 

m + b 

Is j > 0? 

23 


J3Z 

4F 

b 

Is input exhausted? 

24 


IN 

BUFFER (TAPEIN) 

b- 1 

Sentinel sensed; read another 

25 


JBUS 

* (TAPEIN) 


tape block, wait for completion. 

26 


ENT5 

BUFFER 

b- 1 

Reset the buffer pointer. 

27 


JMP 

2B 

b- 1 


28 

3H 

LD4 

1,5 

m 

T3. Record the relation. 

29 


LDA 

X, 4 (COUNT) 

m 

COUNT [fc] 

30 


INCA 

1 

771 

+ 1 

31 


STA 

X, 4 (COUNT) 

m 

-» COUNT [fc] . 

32 


INC2 

1 

771 

AVAIL <- AVAIL + 1. 

33 


LDA 

X,3(T0P) 

771 

TOP [j] 

34 


STA 

0,2 (NEXT) 

771 

— I NEXT (P) . 

35 


ST4 

0,2 (SUC) 

771 

k -> SUC(P). 

36 


ST2 

X,3(T0P) 

771 

P — 1 TOP [ j] . 

37 


INC5 

2 

771 

Increase buffer pointer. 

38 


JMP 

2B 

771 


39 

4H 

IOC 

0 (TAPEIN) 

1 

Rewind the input tape. 

40 


ENT4 

0,6 

1 

T4. Scan for zeros, k <— n. 

41 


ENT5 

-100 

1 

Reset buffer pointer for output. 

42 


ENT3 

0 

1 

R i — 0. 

43 

4H 

LDA 

X, 4 (COUNT) 

71 

Examine COUNT [fc] . 

44 


JAP 

*+3 

71 

Is it nonzero? 

45 


ST4 

X,3(QLINK) 

a 

QLINK [R] <- k. 

46 


ENT 3 

0,4 

a 

R i — k . 

47 


DEC4 

1 

71 


48 


J4P 

4B 

71 

n > k > 1. 
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49 * SORTING PHASE 


50 


LD1 

X(QLINK) 

1 

F <- QLINKI0I. 

51 

5H 

JBUS 

* (TAPEOUT) 


T5. Output front of aueue. 

52 


ST1 

BUFFER+100,5 

n + 1 

Store F in buffer area. 

53 


J1Z 

8F 

n + 1 

Is F zero? 

54 


INC5 

1 

n 

Advance buffer pointer. 

55 


J5N 

*+3 

n 

Test if buffer is full. 

56 


OUT 

BUFFER (TAPEOUT) 

c — 1 

If so, output a tape block. 

57 


ENT 5 

-100 

c- 1 

Reset the buffer pointer. 

58 


DEC6 

1 

n 

N <— N — 1. 

59 


LD2 

X , 1 (TOP) 

n 

P «- TOP IF] . 

60 


J2Z 

7F 

n 

T6. Erase relations. 

61 

6H 

LD4 

0,2 (SUC) 

m 

rI4 <- SUC(P). 

62 


LDA 

X, 4 (COUNT) 

m 

COUNT [rI4] 

63 


DECA 

1 

rn 

-1 

64 


STA 

X, 4 (COUNT) 

m 

-t COUNT [rI4] . 

65 


JAP 

*+3 

m 

Has zero been reached? 

66 


ST4 

X,3(QLINK) 

n — a 

If so, set QLINKIR] C- rI4. 

67 


ENT3 

0,4 

n — a 

R <r- rI4. 

68 


LD2 

0,2 (NEXT) 

m 

P C- NEXT (P) . 

69 


J2P 

6B 

m 

If P / A, repeat. 

70 

7H 

LD1 

X.l(QLINK) 

n 

T7. Remove from queue. 

71 


JMP 

5B 

n 

F <- QLINK(F) , go to T5. 

72 

8H 

OUT 

BUFFER (TAPEOUT) 

1 

T8. End of process. 

73 


IOC 

0 (TAPEOUT) 

1 

Output last block and rewind 

74 


HLT 

0,6 

1 

Stop, displaying N on console. 

75 

X 

END 

T0PS0RT 


Beginning of table area | 


The analysis of Algorithm T is quite simple with the aid of Kirchhoff 's law; 
the execution time has the approximate form cim + c 2 n, where m is the number 
of input relations, n is the number of objects, and C\ and c 2 are constants. It 
is hard to imagine a faster algorithm for this problem! The exact quantities in 
the analysis are given with Program T above, where a — number of objects with 
no predecessor, b = number of tape records in input = [(to + 2)/50]. and c = 
number of tape records in output = \(n + 1)/100]. Exclusive of input-output 
operations, the total running time in this case is only (32m + 24n + 76 + 2c+ 16) ri. 

A topological sorting technique similar to Algorithm T (but without the 
important feature of the queue links) was first published by A. B. Kahn, CACM 5 
(1962), 558—562. The fact that topological sorting of a partial ordering is always 
possible was first proved in print by E. Szpilrajn, Fundamenta Mathematics 16 
(1930), 386-389; he proved it for infinite sets as well as finite sets, and mentioned 
that the result was already known to several of his colleagues. 

I11 spite of the fact that Algorithm T is so efficient, we will study an even 
better algorithm for topological sorting in Section 7.4.1. 

EXERCISES 

► 1 . [10] Operation (9) for popping up a stack mentions the possibility of UNDERFLOW: 

why doesn’t operation (8), pushing down a stack, mention the possibility of OVERFLOW? 
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2. [22] Write a “general purpose” MIX subroutine to do the insertion operation. (10). 
This subroutine should have the following specifications (as in Section 1.4.1): 

Calling secjuence: JMP INSERT Jump to subroutine. 

NOP T Location of pointer variable 

Entry conditions: rA = information to be put into the INFO field of a new node. 

Exit conditions: The stack whose pointer is the link variable T has the new node on 
top; rll = T; rI2, rI3 are altered. 

3. [22] Write a “general purpose” MIX subroutine to do the deletion operation, (n). 
This subroutine should have the following specifications: 

Calling sequence: JMP DELETE Jump to subroutine. 

NOP T Location of pointer variable 

JMP UNDERFLOW First exit, if UNDERFLOW sensed 
Entry conditions: None 

Exit conditions: If the stack whose pointer is the link variable T is empty, the first 
exit is taken; otherwise the top node of that stack is deleted, and exit 
is made to the third location following “JMP DELETE”. In the latter 
case, rll — T and rA is the contents of the INFO field of the deleted 
node. In either case, rI2 and rI3 are used by this subroutine. 

4. [22] The program in (10) is based on the operation P 4= AVAIL, as given in 
(6). Show how to write an OVERFLOW subroutine so that, without any change in the 
coding (10), the operation P 4= AVAIL makes use of SEQMIN, as given by (7). For general- 
purpose use, your subroutine should not change the contents of any registers, except 
rJ and possibly the comparison indicator. It should exit to location rj - 2, instead of 
the usual rJ. 

► 5. [24] Operations (14) and (17) give the effect of a queue; show how to define the 
further operation “insert at front” so as to obtain all the actions of an output-restricted 
deque. How could the operation “delete from rear” be defined (so that we would have 
a general deque)? 

6. [21] In operation (14) we set LINK(P) t— A, while the very next insertion at the 
rear of the queue will change the value of this same link field. Show how the setting of 
LINK(P) in (14) could be avoided if we make a change to the testing of “F = A” in (17). 

► 7. [23] Design an algorithm to “invert” a linked linear list such as (1), that is, to 
change its links so that the items appear in the opposite order. [If, for example, the 
list (1) were inverted, we would have FIRST linking to the node containing item 5; that 
node would link to the one containing item 4; etc.] Assume that the nodes have the 
form (3). 

8. [24] Write a MIX program for the problem of exercise 7, attempting to design your 
program to operate as fast as possible. 

9. [20] Which of the following relations is a partial ordering on the specified set S? 
[Note: If the relation “x A y" is defined below, the intent is to define the relation 

d y = (x A y or x = y),” and then to determine whether A is a partial ordering.] 
(a) S = all rational numbers, x A y means x > y. (b) S = all people, x < y means 
x is an ancestor of y. (c) S = all integers, x A y means a: is a multiple of y (that is, 
x mod y = 0). (d) S = all the mathematical results proved in this book, x A y means 
the proof of y depends upon the truth of x. (e) S = all positive integers, x A y means 
x + y is even, (f ) S = a set of subroutines, x A y means “x calls y” that is, y may be 
in operation while x is in operation, with recursion not allowed. 
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10. [M21] Given that “C” is a relation that satisfies properties (i) and (ii) of a partial 
ordering, prove that the relation defined by the rule “x A y if and only if x = y 
or x C y, satisfies all three properties of a partial ordering. 

► 11. [24] The result of topological sorting is not always completely determined, since 
there may be several ways to arrange the nodes and to satisfy the conditions of 
topological order. Find all possible ways to arrange the nodes of Fig. 6 into topological 
order. 

12. [M20] There are 2 n subsets of a set of n elements, and these subsets are partially 
ordered by the set-inclusion relation. Give two interesting ways to arrange these subsets 
in topological order. 

13. [M 48 ] How many ways are there to arrange the 2 n subsets described in exercise 12 
into topological order? (Give the answer as a function of n.) 

14. [M21] A linear ordering of a set S, also called a total ordering , is a partial ordering 
that satisfies the additional “comparability” condition 

(iv) For any two objects x, y in 5, either x A y or y A x. 

Prove directly from the definitions given that a topological sort can result in only one 
possible output if and only if the relation A is a linear ordering. (You may assume that 
the set S is finite.) 

15. [M25] Show that for any partial ordering on a finite set S there is a unique set 
of irredundant relations that characterizes this ordering, as in ( 18 ) and Fig. 6 . Is the 
same fact true also when S is an infinite set? 

16. [M22] Given any partial ordering on a set S = {xi, . . . , x n }, we can construct its 
incidence matrix ( aij ), where ay = 1 if x t A Xj , and ay = 0 otherwise. Show that 
there is a way to permute the rows and columns of this matrix so that all entries below 
the diagonal are zero. 

► 17. [21] What output does Algorithm T produce if it is presented with the input ( 18 )? 

18. [20] What, if anything, is the significance of the values of QLINK [0] , QLINK [1] . 
.... QLINK [n] when Algorithm T terminates? 

19. [?£] In Algorithm T we examine the front position of the queue in step T 5 . but 
do not remove that element from the queue until step T7. What would happen if we 
set F ■(- QLINK [F] at the conclusion of step T5, instead of in T7? 

►20. [24] Algorithm T uses F, R, and the QLINK table to obtain the effect of a queue that 
contains those nodes whose COUNT field has become zero but whose successor relations 
have not yet been removed. Could a stack be used for this purpose instead of a queue? 
If so, compare the resulting algorithm with Algorithm T. 

21. [21] Would Algorithm T still perform a valid topological sort if one of the relations 
“j A fc” were repeated several times in the input? What if the input contained a relation 
of the form “j j”? 

22. [23] Program T assumes that its input tape contains valid information, but a 
program that is intended for general use should always make careful tests on its input so 
that clerical errors can be detected, and so that the program cannot “destroy itself.” For 
example, if one of the input relations for k were negative, Program T may erroneously 
change one of its own instructions when storing into X [fc] . Suggest ways to modify 
Program T so that it is suitable for general use. 
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► 23 . [27] When the topological sort algorithm cannot proceed because it has detected 
a loop in the input (see step T8), it is usually of no use to stop and say, “There was 
a loop.’’ It is helpful to print out one of the loops, thereby showing part of the input 
that was in error. Extend Algorithm T so that it will do this additional printing of a 
loop when necessary. [Hint: The text gives a proof for the existence of a loop when 
N > 0 in step T8; that proof suggests an algorithm.] 

24 . [24] Incorporate the extensions of Algorithm T made in exercise 23 into Pro- 
gram T. 

25 . [4 7] Design as efficient an algorithm as possible for doing a topological sort of very 
large sets S having considerably more nodes than the computer memory can contain. 
Assume that the input, output, and temporary working space are done with magnetic 
tape. [Possible hint: A conventional sort of the input allows us to assume that all 
relations for a given node appear together. But then what can be done? In particular, 
we must consider the worst case in which the given ordering is already a linear ordering 
that has been wildly permuted; exercise 24 in the introduction to Chapter 5 explains 
how to handle this case with O(logn) 2 passes over the data.] 

26 . [29] (Subroutine allocation.) Suppose that we have a tape containing the main 
subroutine library in relocatable form, for a 1960s-style computer installation. The 
loading routine wants to determine the amount of relocation for each subroutine used, 
so that it can make one pass through the tape to load the necessary routines. The 
problem is that some subroutines require others to be present in memory. Infrequently 
used subroutines (which appear toward the end of the tape) may call on frequently 
used subroutines (which appear toward the beginning of the tape), and we want to 
know all of the subroutines that are required, before passing through the tape. 

One way to tackle this problem is to have a “tape directory” that fits in memory. 
The loading routine has access to two tables: 

a) The tape directory. This table is composed of variable-length nodes having 
the form 


B 

SPACE 

LINK 

B 

SUB1 

SUB2 


B 

SPACE 

LINK 

B 

SUB1 

SUB2 


B 

SUBn 

6 


' 

B 

SUB(n-l) 

SUBn 


where SPACE is the number of words of memory required by the subroutine; LINK is a 
link to the directory entry for the subroutine that appears on the tape following this 
subroutine; SUB1, SUB2, ..., SUBrt (n > 0) are links to the directory entries for any 
other subroutines required by this one; B = 0 on all words except the last, B = -1 on 
the last word of a node. The address of the directory entry for the first subroutine on 
the library tape is specified by the link variable FIRST. 

b) The list of subroutines directly referred to by the program to be loaded. This is 
stored in consecutive locations X [1] , X [2] , . . . , X [N] , where N > 0 is a variable known 
to the loading routine. Each entry in this list is a link to the directory entry for the 
subroutine desired. 

The loading routine also knows MLOC, the amount of relocation to be used for the 
first subroutine loaded. 
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As a small example, consider the following configuration: 



Tape 

directory 


List of subroutines needed 


B 

SPACE 

LINK 

X [1] = 1003 

1000 

0 

20 

1005 

X [2] = 1010 

1001 

-1 

1002 

0 


1002 

-1 

30 

1010 

N = 2 

1003 

0 

200 

1007 

FIRST = 1002 

1004 

-1 

1000 

1006 

ML0C = 2400 

1005 

-1 

100 

1003 


1006 

-1 

60 

1000 


1007 

0 

200 

0 


1008 

0 

1005 

1002 


1009 

-1 

1006 

0 


1010 

-1 

20 

1006 



The tape directory in this case shows that the subroutines on tape are 1002, 1010, 
1006, 1000, 1005, 1003, and 1007 in that order. Subroutine 1007 takes 200 locations 
and implies the use of subroutines 1005, 1002, and 1006; etc. The program to be loaded 
requires subroutines 1003 and 1010, which are to be placed into locations > 2400. These 
subroutines in turn imply that 1000, 1006, and 1002 must also be loaded. 

The subroutine allocator is to change the X-table so that each entry X [1] , X [2] , 
X [3] , ... has the form 



(except the last entry, which is explained below), where SUB is a subroutine to be loaded, 
and BASE is the amount of relocation. These entries are to be in the order in which the 
subroutines appear on tape. One possible answer for the example above would be 



BASE 

SUB 


BASE 

SUB 

X [1] : 

2400 

1002 

X [4] : 

2510 

1000 

X [2] : 

2430 

1010 

X [5] : 

2530 

1003 

X [3] : 

2450 

1006 

X [6] : 

2730 

0 


The last entry contains the first unused memory address. 

(Clearly, this is not the only way to treat a library of subroutines. The proper way 
to design a library is heavily dependent upon the computer used and the applications 
to be handled. Large modern computers require an entirely different approach to 
subroutine libraries. But this is a nice exercise anyway, because it involves interesting 
manipulations on both sequential and linked data.) 

The problem in this exercise is to design an algorithm for the stated task. Your 
allocator may transform the tape directory in any way as it prepares its answer, 
since the tape directory can be read in anew by the subroutine allocator on its next 
assignment, and the tape directory is not needed by other parts of the loading routine. 

27. [25] Write a MIX program for the subroutine allocation algorithm of exercise 26. 

28. [40] The following construction shows how to “solve” a fairly general type of two- 
person game, including chess, nim, and many simpler games: Consider a finite set of 
nodes, each of which represents a possible position in the game. For each position there 
are zero or more moves that transform that position into some other position. We say 
that position x is a predecessor of position y (and y is a successor of x) if there is a 
move from x to y. Certain positions that have no successors are classified as won or 
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lost positions. The player to move in position x is the opponent of the player to move 
in the successors of position x. 

Given such a configuration of positions, we can compute the complete set of won 
positions (those in which the next player to move can force a victory) and the complete 
set of lost positions (those in which the player must lose against an expert opponent) 
by repeatedly doing the following operation until it yields no change: Mark a position 
"lost” if all its successors are marked “won”; mark a position “won” if at least one of 
its successors is marked “lost.” 

After this operation has been repeated as many times as possible, there may be 
some positions that have not been marked at all; a player in such a position can neither 
force a victory nor be compelled to lose. 

This procedure for obtaining the complete set of won and lost positions can be 
adapted to an efficient algorithm for computers that closely resembles Algorithm T. 
We may keep with each position a count of the number of its successors that have not 
been marked “won,” and a list of all its predecessors. 

The problem in this exercise is to work out the details of the algorithm that has 
just been so vaguely described, and to apply it to some interesting games that do not 
involve too many possible positions [like the “military game”: E. Lucas, Recreations 
Mathematiques 3 (Paris: 1893) 105-116; E. R. Berlekamp, J. H. Conway, and R. K. 
Guy, Winning Ways 3 (A. K. Peters, 2003), Chapter 21]. 

► 29. [21] (a) Give an algorithm to “erase” an entire list like (l), by putting all of its 
nodes on the AVAIL stack, given only the value of FIRST. The algorithm should operate 
as fast as possible, (b) Repeat part (a) for a list like ( 12 ), given the values of F and R. 
30. [17] Suppose that queues are represented as in ( 12 ), but with an empty queue 
represented by F = A and R undefined. What insertion and deletion procedures should 
replace ( 14 ) and ( 17 )? 

2.2.4. Circular Lists 

A slight change in the manner of linking furnishes us with an important alter- 
native to the methods of the preceding section. 

A circularly linked list (briefly: a circular list) has the property that its last 
node links back to the first instead of to A. It is then possible to access all of 
the list starting at any given point; we also achieve an extra degree of symmetry, 
and if we choose we need not think of the list as having a last or first node. 

The following situation is typical: 



Assume that the nodes have two fields, INFO and LINK, as in the preceding 
section. There is a link variable PTR that points to the rightmost node of the 
list, and LINK (PTR) is the address of the leftmost node. The following primitive 
operations are most important: 

a) Insert Y at left: P <= AVAIL, INFO(P) t- Y, LINK(P) LINK (PTR), 
LINK(PTR) P. 

b) Insert Y at right: Insert Y at left, then PTR <— P. 

c) Set Y to left node and delete: P «- LINK (PTR) , Y <- INFO(P), LINK (PTR) «- 
LINK(P), AVAIL P. 
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Operation (b) is a little surprising at first glance; the operation PTR 4— LINK(PTR) 
effectively moves the leftmost node to the right in the diagram (1), and this is 
quite easy to understand if the list is regarded as a circle instead of as a straight 
line with connected ends. 

The alert reader will observe that we have made a serious mistake in oper- 
ations (a), (b), and (c). What is it? Answer: We have forgotten to consider the 
possibility of an empty list. If, for example, operation (c) is applied five times 
to the list ( 1 ), we will have PTR pointing to a node in the AVAIL list, and this 
can lead to serious difficulties; for example, imagine applying operation (c) once 
more! If we take the position that PTR will equal A in the case of an empty 
list, we could remedy the operations by inserting the additional instructions “if 
PTR = A, then PTR 4— LINK(P) 4- P; otherwise ...” after “INFO(P) 4— Y” in (a); 
preceding (c) by the test “if PTR = A, then UNDERFLOW”; and following (c) by “if 
PTR = P, then PTR 4- A”. 

Notice that operations (a), (b), and (c) give us the actions of an output- 
restricted deque, in the sense of Section 2.2.1. Therefore we find in particular 
that a circular list can be used as either a stack or a queue. Operations (a) 
and (c) combined give us a stack; operations (b) and (c) give us a queue. These 
operations are only slightly less direct than their counterparts in the previous 
section, where we saw that operations (a), (b), and (c) can be performed on 
linear lists using two pointers F and R. 

Other important operations become efficient with circular lists. For example, 
it is very convenient to “erase” a list, that is, to put an entire circular list onto 
the AVAIL stack at once: 

If PTR ^ A, then AVAIL 44- LINK (PTR). ( 2 ) 

[Recall that the “4-4” operation denotes interchange: P 4— AVAIL, AVAIL 4— 
LINK (PTR), LINK (PTR) 4- P.] Operation (2) is clearly valid if PTR points any- 
where in the circular list. Afterwards we should of course set PTR 4— A. 

Using a similar technique, if PTRi and PTR2 point to disjoint circular lists Li 
and L 2 , respectively, we can insert the entire list L 2 at the right of L x : 

If PTR 2 / A, then 

(if PTR X ^ A, then LINK (PTR X ) 44 LINK(PTR 2 ); (3) 

set PTRj 4- PTR 2 , PTR 2 4- A). 

Splitting one circular list into two, in various ways, is another simple opera- 
tion that can be done. These operations correspond to the concatenation and 
deconcatenation of strings. 

Thus we see that a circular list can be used not only to represent inherently 
circular structures, but also to represent linear structures; a circular list with one 
pointer to the rear node is essentially equivalent to a straight linear list with two 
pointers to the fiont and rear. The natural question to ask, in connection with 
this observation, is ‘How do we find the end of the list, when there is circular 
symmetry? There is no A link to signal the end! The answer is that when we 
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are operating on an entire list, moving from one node to the next, we should stop 
when we get back to our starting place (assuming, of course, that the starting 
place is still present in the list). 

An alternative solution to the problem just posed is to put a special, recog- 
nizable node into each circular list, as a convenient stopping place. This special 
node is called the list head, and in applications we often find it is quite convenient 
to insist that every circular list have exactly one node that is its list head. One 
advantage is that the circular list will then never be empty. With a list head, 
diagram ( 1 ) becomes 


List head 




(4) 


References to lists like ( 4 ) are usually made via the list head, which is often in a 
fixed memory location. The disadvantage of list heads is that there is no pointer 
to the right end, so we must sacrifice operation (b) stated above. 

Diagram (4) may be compared with 2.2.3-(i) at the beginning of the previ- 
ous section, in which the link associated with “item 5” now points to LOC (FIRST) 
instead of to A; the variable FIRST is now thought of as a link within a node, 
namely the link that is in NDDE(LOC(FIRST) ) . The principal difference between 
(4) and 2.2.3-(i) is that (4) makes it possible (though not necessarily efficient) 
to get to any point of the list from any other point. 

As an example of the use of circular lists, we will discuss arithmetic on 
polynomials in the variables x, y , and z, with integer coefficients. There are 
many problems in which a scientist wants to manipulate polynomials instead of 
just numbers; we are thinking of operations like the multiplication of 

(x 4 + 2x 3 y + 3x 2 y 2 + 4xy 3 + 5y 4 ) by (x 2 — 2 xy + y 2 ) 


to get 

(x 6 — 6 xy 5 + 5 y 6 ). 


Linked allocation is a natural tool for this purpose, since polynomials can grow to 
unpredictable sizes and we may want to represent many polynomials in memory 
at the same time. 

We will consider here the two operations of addition and multiplication. Let 
us suppose that a polynomial is represented as a list in which each node stands 
for one nonzero term, and has the two-word form 


COEF 

± 

A 

B 

C 

LINK 


(5) 


Here COEF is the coefficient of the term in x A y B z c . We will assume that the 
coefficients and exponents will always lie in the range allowed by this format, and 
that it is not necessary to check the ranges during our calculations. The notation 
ABC will be used to stand for the ± A B C fields of the node ( 5 ), treated as a single 
unit. The sign of ABC, namely the sign of the second word in ( 5 ), will always be 
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plus, except that there is a special node at the end of every polynomial that has 
ABC = — 1 and COEF = 0. This special node is a great convenience, analogous to 
our discussion of a list head above, because it provides a convenient sentinel and 
it avoids the problem of an empty list (corresponding to the polynomial 0). The 
nodes of the list always appear in decreasing order of the ABC field, if we follow 
the direction of the links, except that the special node (which has ABC = -1) 
links to the largest value of ABC. For example, the polynomial x 6 - 6 xy 5 + 5 y 6 
would be represented thus: PTR 



Algorithm A ( Addition of polynomials). This algorithm adds polynomial(P) 
to polynomial (Q), assuming that P and Q are pointer variables pointing to 
polynomials having the form above. The list P will be unchanged; the list q 
will retain the sum. Pointer variables P and Q return to their starting points 
at the conclusion of this algorithm; auxiliary pointer variables Q1 and Q2 are 
also used. 

Al. [Initialize.] Set P <- LINK(P), Q1 4- Q, Q 4- LINK(Q) . (Now both P and Q 
point to the leading terms of their polynomials. Throughout most of this 
algorithm the variable Q1 will be one step behind Q, in the sense that Q = 
LINK(Ql).) 

A2. [ABC(P) : ABC(Q) .] If ABC(P) < ABC(Q) , set Q 1 -e— Q and Q 4- LINK(Q) and 
repeat this step. If ABC(P) = ABC (Q) , go to step A3. If ABC(P) > ABC(Q). 
go to step A5. 

A3. [Add coefficients.] (We’ve found terms with equal exponents.) If ABC (P) < 0. 
the algorithm terminates. Otherwise set COEF(Q) 4- COEF(Q) + COEF(P). 
Now if COEF (Q) = 0, go to A4; otherwise, set P 4- LINK(P), Q1 4- Q. 
Q 4 - LINK(Q), and go to A2. (Curiously the latter operations are identical 
to step Al.) 

A4. [Delete zero term.] Set Q2 4- Q, LINK(Ql) 4- Q 4- LINK(Q) , and AVAIL 
Q2. (A zero term created in step A3 has been removed from polynomial (Q) .) 
Set P 4— LINK(P) and go back to A2. 

A5. [Insert new term.] ( Polynomial (P) contains a term that is not present 
in polynomial (Q) , so we insert it in polynomial(Q).) Set Q2 4 = AVAIL. 
C0EFCQ2) 4- COEF (P) , ABC (Q2) 4- ABC (P) , LINK (Q2) 4- Q, LINK(Ql) 4- Q2. 
Q1 4— Q2, P 4— LINK(P), and return to step A2. | 

One of the most noteworthy features of Algorithm A is the manner in which 
the pointer variable Q1 follows the pointer q around the list. This is very typical 
of list processing algorithms, and we will see a dozen more algorithms with the 
same characteristic. Can the reader see why this idea was used in Algorithm A? 

A reader who has little prior experience with linked lists will find it very 
instructive to study Algorithm A carefully; as a test case, try adding x + y + z 
to x 2 — 2 y — z. 
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Given Algorithm A, the multiplication operation is surprisingly easy: 

Algorithm M ( Multiplication of polynomials). This algorithm, analogous to 
Algorithm A, replaces polynomial (Q) by 

polynomial(Q) + polynomial(M) x polynomial (P) . 

Ml. [Next multiplier.] Set M «— LINK(M). If ABC(M) < 0, the algorithm termi- 
nates. 

M2. [Multiply cycle.] Perform Algorithm A, except that wherever the notation 
“ABC(P)” appears in that algorithm, replace it by “(if ABC(P) < 0 then — 1, 
otherwise ABC(P) + ABC(M))”; wherever “COEF(P)” appears in that algo- 
rithm replace it by “COEF(P) x COEF(M)”. Then go back to step Ml. | 

The programming of Algorithm A in MIX language shows again the ease with 
which linked lists are manipulated in a computer. In the following code we 
assume that OVERFLOW is a subroutine that either terminates the program (due 
to lack of memory space) or finds further available space and exits to rJ — 2. 

Program A ( Addition of polynomials). This is a subroutine written so that it 
can be used in conjunction with a multiplication subroutine (see exercise 15). 

Calling sequence: JMP ADD 
Entry conditions: rll = P, rI2 = Q. 

Exit conditions: polynomial (Q) has been replaced by polynomial (Q) + poly- 
nomial (P); rll and rI2 are unchanged; all other registers have 
undefined contents. 

In the coding below, P = rll, Q = rI2, Q1 = rI3, and Q2 = rI6, in the notation of 
Algorithm A. 


01 

LINK 

EQU 

4:5 


Definition of LINK field 

02 

ABC 

EQU 

0:3 


Definition of ABC field 

03 

ADD 

STJ 

3F 

1 

Entrance to subroutine 

04 

1H 

ENT3 

0,2 

1 + m" 

Al. Initialize. Set Q1 <— Q. 

05 


LD2 

1,3 (LINK) 

1 + m" 

Q 4- LINK(Ql) . 

06 

OH 

LD1 

1,1 (LINK) 

1 +p 

P -f- LINK(P) . 

07 

SW1 

LDA 

1,1 

1 +p 

rA(0:3) 4- ABC(P). 

08 

2H 

CMPA 

1,2 (ABC) 

X 

A2. ABC(P) :ABC(Q) . 

09 


JE 

3F 

X 

If equal, go to A3. 

10 


JG 

5F 

p' + q' 

If greater, go to A5. 

11 


ENT3 

0,2 

q' 

If less, set Q1 4— Q. 

12 


LD2 

1,3 (LINK) 

q 

q 4- LINK(Ql) . 

13 


JMP 

2B 

q' 

Repeat. 

14 

3H 

JAN 

* 

771 + 1 

A3. Add coefficients. 

15 

SW2 

LDA 

0,1 

m 

C0EF(P) 

16 


ADD 

0,2 

m 

+ C0EF (Q) 

17 


STA 

0,2 

771 

— > C0EF(q). 

18 


JANZ 

IB 

777 

Jump if nonzero. 
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19 ENT6 0,2 

20 LD2 1,2 (LINK) 

21 LDX AVAIL 

22 STX 1,6 (LINK) 

23 ST6 AVAIL 

24 ST2 1,3 (LINK) 

25 JMP OB 

26 5H LD6 AVAIL 

27 J6Z OVERFLOW 

28 LDX 1,6 (LINK) 

29 STX AVAIL 

30 STA 1,6 

31 SW3 LDA 0,1 

32 STA 0,6 

33 ST2 1,6 (LINK) 

34 ST6 1,3 (LINK) 

35 ENT3 0,6 

36 JMP OB 

Note that Algorithm A traverses each of the two lists just once; it is not 
necessary to loop around several times. Using Kirchhoff’s law, we find that 
an analysis of the instruction counts presents no difficulties; the execution time 
depends on four quantities 

m! = number of matching terms that cancel with each other; 
m" = number of matching terms that do not cancel; 
p' — number of unmatched terms in polynomial (P); 
q' ~ number of unmatched terms in polynomial (Q) . 

The analysis given with Program A uses the abbreviations 

m = m' + m", p = m + p 1 , q = m + q ’ , x = 1 + to + p’ + q'; 

the running time for MIX is (27m' + 18m" + 27 p’ + 89' + 13)u. The total number 
of nodes in the storage pool needed during the execution of the algorithm is at 
least 2 + p + q, and at most 2 + p + q + p 1 . 

EXERCISES 

1. [21] The text suggests at the beginning of this section that an empty circular list 
could be represented by PTR = A. It might be more consistent with the philosophy 
of circular lists to have PTR = LOC(PTR) indicate an empty list. Does this convention 
facilitate operations (a), (b), or (c) described at the beginning of this section? 

2. [20] Draw “before and after” diagrams illustrating the effect of the concatenation 
operation (3), assuming that PTRi and PTR 2 are ^ A. 

► 3. [20] What does operation (3) do if PTRi and PTR 2 are both pointing to nodes in 

the same circular list? 

4. [20] State insertion and deletion operations that give the effect of a stack, using 
representation (4). 


TO 

to ' 

to ' 

to ' 

m! 

to ' 

to ,' 

P' 

p’ 

P 

P 

P 1 

P 1 

P' 

P 

P 1 

P 1 

P 1 


A4. Delete zero term. Q2 
Q 4- LINK(Q) . 


AVAIL 


Q2. 


LINK(Ql) 4- Q. 

Go to advance P. 

A5. Insert new term. 


> Q2 <= AVAIL. 


ABC(Q2) 4- ABC(P) . 
rA 4- COEF(P) . 
C0EF(Q2) 4- rA. 
LINK(Q2) 4- q. 
LINK(qi) 4- q2. 
qi 4- q 2 . 

Go to advance P. £ 
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Fig. 10. Addition of polynomials. 


► 5. [21] Design an algorithm that takes a circular list such as (l) and reverses the 
direction of all the arrows. 

6. [IS] Give diagrams of the list representation for the polynomials (a) xz — 3; (b) 0. 

7. [10] Why is it useful to assume that the ABC fields of a polynomial list appear in 
decreasing order? 

► 8. [10] Why is it useful to have Q1 trailing one step behind Q in Algorithm A? 

► 9. [23] Would Algorithm A work properly if P = Q (i.e., both pointer variables point 
at the same polynomial)? Would Algorithm M work properly if P = M, if P = Q, or if 
M = Q? 

► 10. [20] The algorithms in this section assume that we are using three variables x, y, 
and ^ in the polynomials, and that their exponents individually never exceed 6—1 
(where b is the byte size in Mix’s case). Suppose instead that we want to do addition 
and multiplication of polynomials in only one variable, x, and to let its exponent take 
on values up to 6 3 — 1. What changes should be made to Algorithms A and M? 

11. [24] (The purpose of this exercise and many of those following is to create a pack- 
age of subroutines useful for polynomial arithmetic, in conjunction with Program A.) 
Since Algorithms A and M change the value of polynomial (Q), it is sometimes desirable 
to have a subroutine that makes a copy of a given polynomial. Write a MIX subroutine 
with the following specifications: 

Calling sequence: JMP COPY 
Entry conditions: rll = P 

Exit conditions: rI2 points to a newly created polynomial equal to polynomial (P) ; 
rll is unchanged; other registers are undefined. 

12. [21] Compare the running time of the program in exercise 11 with that of Pro- 
gram A when polynomial (0) = 0. 

13. [20] Write a MIX subroutine with the following specifications: 

Calling sequence: JMP ERASE 
Entry conditions: rll = P 

Exit conditions: polynomial (P) has been added to the AVAIL list; all register contents 
are undefined. 

[Note: This subroutine can be used in conjunction with the subroutine of exercise 11 
in the sequence “LD1 Q; JMP ERASE; LD1 P; JMP COPY; ST2 Q” to achieve the effect 
“polynomial (Q) polynomial (P)”.] 
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14. [22] Write a MIX subroutine with the following specifications: 

Calling sequence: JMP ZERO 

Entry conditions: None 

Exit conditions: rI2 points to a newly created polynomial equal to 0; other register 
contents are undefined. 

15. [24] Write a MIX subroutine to perform Algorithm M, having the following specifi- 
cations: 

Calling sequence: JMP MULT 

Entry conditions: rll = P, rI2 = Q, rI4 = M. 

Exit conditions: polynomial(Q) «— polynomial(Q) + polynomial (M) x polynomial (P ) : 

rll, rI2, rI4 are unchanged; other registers undefined. 

[Note: Use Program A as a subroutine, changing the settings of SW1, SW2, and SW3.] 

16. [M28] Estimate the running time of the subroutine in exercise 15 in terms of some 
relevant parameters. 

► 17. [22] What advantage is there in representing polynomials with a circular list as 
in this section, instead of with a straight linear linked list terminated by A as in the 
previous section? 

►18. [25] Devise a way to represent circular lists inside a computer in such a way that 
the list can be traversed efficiently in both directions, yet only one link field is used 
per node. [Hint: If we are given two pointers, to two successive nodes Xi-\ and it 
should be possible to locate both x i+1 and x t _ 2 .] 


2.2.5. Doubly Linked Lists 

For even greater flexibility in the manipulation of linear lists, we can include two 
links in each node, pointing to the items on either side of that node: 


LEFT 


i 


i 


- RIGHT 


(l) 

Here LEFT and RIGHT are pointer variables to the left and right of the list. Each 
node of the list includes two links, called, for example, LLINK and RLINK. The 
operations of a general deque are readily performed with such a representation; 
see exercise 1. However, manipulations of doubly linked lists almost always 
become much easier if a list head node is part of each list, as described in the 
preceding section. When a list head is present, we have the following typical 
diagram of a doubly linked list: 

List head 


£ 




( 2 ) 

The RLINK and LLINK fields of the list head take the place of LEFT and RIGHT 
in (i). There is complete symmetry between left and right; the list head could 
equally well have been shown at the right of ( 2 ). If the list is empty, both link 
fields of the list head point to the head itself. 

The list representation ( 2 ) clearly satisfies the condition 


RLINK (LLINK (X)) = LLINK (RLINK (X) ) =X 


(3) 
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if X is the location of any node in the list (including the head). This fact is the 
principal reason that representation ( 2 ) is preferable to ( 1 ). 

A doubly linked list usually takes more memory space than a singly linked 
one does (although there is sometimes room for another link in a node that 
doesn’t fill a complete computer word). But the additional operations that can be 
performed efficiently with two-way links are often more than ample compensation 
for the extra space requirement. Besides the obvious advantage of being able to 
go back and forth at will when examining a doubly linked list, one of the principal 
new abilities is the fact that we can delete NODE(X) from the list it is in, given 
only the value of X. This deletion operation is easy to derive from a “before and 
after” diagram (Fig. 11) and it is very simple: 

RLINKCLLINK(X)) <- RLINK(X), LLINK(RLINK(X) ) 4- LLINK(X) , 

(4) 

AVAIL <f= X. 


In a list that has only one-way links, we cannot delete NODE(X) without 
knowing which node precedes it in the chain, since the preceding node needs to 
have its link altered when NODE(X) is deleted. In all the algorithms considered in 
Sections 2.2.3 and 2.2.4 this additional knowledge was present whenever a node 
was to be deleted; see, in particular, Algorithm 2.2.4A, where we had pointer Q1 
following pointer Q for just this purpose. But we will meet several algorithms 
that require removing random nodes from the middle of a list, and doubly linked 
lists are frequently used just for this reason. (We should point out that in a 
circular list it is possible to delete NODE(X), given X, if we go around the entire 
circle to find the predecessor of X. But this operation is clearly inefficient when 
the list is long, so it is rarely an acceptable substitute for doubly linking the list. 
See also the answer to exercise 2. 2. 4-8.) 


X 



After JJ. 

AVAIL 


Fig. 11. Deletion from 
a doubly linked list. 


Similarly, a doubly linked list permits the easy insertion of a node adjacent 
to NODE(X) at either the left or the right. The steps 

P <= AVAIL, LLINK(P) 4— X, RLINK(P) 4- RLINK(X) , , , 

LLINK(RLINK(X)) 4- P, RLINK(X)^P 

do such an insertion to the right of NODE(X); and by interchanging left and 
right we get the corresponding algorithm for insertion to the left. Operation 
( 5 ) changes the settings of five links, so it is a little slower than an insertion 
operation in a one-way list where only three links need to be changed. 
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As an example of the use of doubly linked lists, we will now consider the 
writing of a discrete simulation program. “Discrete simulation” means the 
simulation of a system in which all changes in the state of the system may 
be assumed to happen at certain discrete instants of time. The “system” being 
simulated is usually a set of individual activities that are largely independent 
although they interact with each other; examples are customers at a store, ships 
in a harbor, people in a corporation. In a discrete simulation, we proceed by 
doing whatever is to be done at a certain instant of simulated time, then advance 
the simulated clock to the next time when some action is scheduled to occur. 

By contrast, a “continuous simulation” would be simulation of activities that 
are under continuous changes, such as traffic moving on a highway, spaceships 
traveling to other planets, etc. Continuous simulation can often be satisfactorily 
approximated by discrete simulation with very small time intervals between 
steps; however, in such a case we usually have “synchronous” discrete simulation, 
in which many parts of the system are slightly altered at each discrete time 
interval, and such an application generally calls for a somewhat different type of 
program organization than the kind considered here. 

The program developed below simulates the elevator system in the Mathe- 
matics building of the California Institute of Technology. The results of such a 
simulation will perhaps be of use only to people who make reasonably frequent 
visits to Caltech; and even for them, it may be simpler just to try using the 
elevator several times instead of writing a computer program. But, as is usual 
with simulation studies, the methods we will use are of much more interest than 
the answers given by the program. The methods to be discussed below illustrate 
typical implementation techniques used with discrete simulation progr ams 

The Mathematics building has five floors: sub-basement, basement, first, 
second, and third. There is a single elevator, which has automatic controls 
and can stop at each floor. For convenience we will renumber the floors 0, 1, 
2, 3, and 4. 

On each floor there are two call buttons, one for UP and one for DOWN. 
(Actually floor 0 has only UP and floor 4 has only DOWN, but we may ignore 
that anomaly since the excess buttons will never be used.) Corresponding to 
these buttons, there are ten variables CALLUP [j] and CALLDOWN [ j] , 0 < j < 4. 
There are also variables CALLCAR [ j] , 0 < j < 4, representing buttons within 
the elevator car, which direct it to a destination floor. When a person presses a 
button, the appropriate variable is set to 1; the elevator clears the variable to 0 
after the request has been fulfilled. 

So far we have described the elevator from a user’s point of view; the 
situation is more interesting as viewed by the elevator. The elevator is in one of 
three states: GDINGUP, G0INGD0WN, or NEUTRAL. (The current state is indicated 
to passengers by lighted arrows inside the elevator.) If it is in NEUTRAL state and 
not on floor 2, the machine will close its doors and (if no command is given by 
the time its doors are shut) it will change to GOINGUP or G0INGD0WN, heading 
for floor 2. (This is the “home floor,” since most passengers get in there.) On 
floor 2 in NEUTRAL state, the doors will eventually close and the machine will wait 
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silently for another command. The first command received for another floor 
sets the machine GOINGUP or GOINGDOWN as appropriate; it stays in this state 
until there are no commands waiting in the same direction, and then it switches 
direction or switches to NEUTRAL just before opening the doors, depending on 
what other commands are in the CALL variables. The elevator takes a certain 
amount of time to open and close its doors, to accelerate and decelerate, and 
to get from one floor to another. All of these quantities are indicated in the 
algorithm below, which is much more precise than an informal description can 
be. The algorithm we will now study may not reflect the elevator’s true principles 
of operation, but it is believed to be the simplest set of rules that explain all 
the phenomena observed during several hours of experimentation by the author 
during the writing of this section. 

The elevator system is simulated by using two coroutines, one for the pas- 
sengers and one for the elevator; these routines specify all the actions to be 
performed, as well as various time delays that are to be used in the simulation. 
In the following description, the variable TIME represents the current value of 
the simulated time clock. All units of time are given in tenths of seconds. There 
are also several other variables: 

FLOOR, the current position of the elevator; 

Dl, a variable that is zero except during the time people are getting in or 
out of the elevator; 

D2, a variable that becomes zero if the elevator has sat on one floor without 
moving for 30 sec or more; 

D3, a variable that is zero except when the doors are open but nobody is 
getting in or out of the elevator; 

STATE, the current state of the elevator (GOINGUP, GOINGDOWN, or NEUTRAL). 

Initially FLOOR = 2, Dl = D2 = D3 = 0, and STATE = NEUTRAL. 

Coroutine U (Users). Everyone who enters the system begins to perform the 
actions specified below, starting at step Ul. 

Ul. [Enter, prepare for successor.] The following quantities are determined in 
some manner that will not be specified here: 

IN, the floor on which the new user has entered the system; 

OUT, the floor to which this user wants to go (OUT ^ IN); 

GIVEUPTIME, the amount of time this user will wait for the elevator before 
running out of patience and deciding to walk; 

INTERTIME, the amount of time before another user will enter the system. 

After these quantities have been computed, the simulation program sets 
things up so that another user enters the system at TIME + INTERTIME. 

U2. [Signal and wait.] (The purpose of this step is to call for the elevator; some 
special cases arise if the elevator is already on the right floor.) If FLOOR = IN 
and if the elevator’s next action is step E6 below (that is, if the elevator doors 
are now closing), send the elevator immediately to its step E3 and cancel its 
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activity E6. (This means that the doors will open again before the elevator 
moves.) If FLOOR = IN and if D3 0, set D3 <- 0, set D1 to a nonzero value, 
and start up the elevator’s activity E4 again. (This means that the elevator 
doors are open on this floor, but everyone else has already gotten on or 
off. Elevator step E4 is a sequencing step that grants people permission to 
enter the elevator according to normal laws of courtesy; therefore, restarting 
E4 gives this user a chance to get in before the doors close.) In all other 
cases, the user sets CALLUP [IN] <- 1 or CALLDOWN [IN] <- 1, according as 
OUT > IN or OUT < IN; and if D2 = 0 or the elevator is in its “dormant” 
position El, the DECISION subroutine specified below is performed. (The 
DECISION subroutine is used to take the elevator out of NEUTRAL state at 
certain critical times.) 

U3. [Enter queue.] Insert this user at the rear of QUEUE [IN] , which is a linear 
list representing the people waiting on this floor. Now the user waits 
patiently for GIVEUPTIME units of time, unless the elevator arrives first — 
more precisely, unless step E4 of the elevator routine below sends this user 
to U5 and cancels the scheduled activity U4. 

U4. [Give up.] If FLOOR ^ IN or D1 = 0, delete this user from QUEUE [IN] 
and from the simulated system. (The user has decided that the elevator is 
too slow, or that a bit of exercise will be better than an elevator ride.) If 
FLOOR = IN and D1 ^ 0, the user stays and waits (knowing that the wait 
won’t be long). 

U5. [Get in.] This user now leaves QUEUE [IN] and enters ELEVATOR, which is 
a stack-like list representing the people now on board the elevator Set 

CALLCAR[0UT] <- 1. 

Now if STATE = NEUTRAL, set STATE <- GOINGUP or G0INGD0WN as 
appropriate, and set the elevator’s activity E5 to be executed after 25 units 
of time. (This is a special feature of the elevator, allowing the doors to close 
faster than usual if the elevator is in NEUTRAL state when the user selects a 
destination floor. The 25-unit time interval gives step E4 the opportunity 
to make sure that D1 is properly set up by the time step E5, the door-closing 
action, occurs.) 

Now the user waits until being sent to step U6 by step E4 below, when 
the elevator has reached the desired floor. 

U6. [Get out.] Delete this user from the ELEVATOR list and from the simulated 
system. | 

Coroutine E (Elevator). This coroutine represents the actions of the elevator; 

step E4 also handles the control of when people get in and out. 

El. [Wait for call.] (At this point the elevator is sitting at floor 2 with the doors 
closed, waiting for something to happen.) If someone presses a button, the 
DECISION subroutine will take us to step E3 or E6. Meanwhile, wait. 

E2. [Change of state?] If STATE = GOINGUP and CALLUP [ j] = CALLDOWN [j] = 
CALLCAR [j] = 0 for all j > FLOOR, then set STATE i- NEUTRAL or STATE <- 
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GOINGDOWN, according as CALLCAR [ j] = 0 for all j < FLOOR or not, and set 
all CALL variables for the current floor to zero. If STATE = GOINGDOWN, do 
similar actions with directions reversed. 

E3. [Open doors.] Set D1 and D2 to any nonzero values. Set elevator activity 
E9 to start up independently after 300 units of time. (This activity may be 
canceled in step E6 below before it occurs. If it has already been scheduled 
and not canceled, we cancel it and reschedule it.) Also set elevator activity 
E5 to start up independently after 76 units of time. Then wait 20 units of 
time (to simulate opening of the doors) and go to E4. 

E4. [Let people out, in.] If anyone in the ELEVATOR list has OUT = FLOOR, send 
the user of this type who has most recently entered immediately to step U6, 
wait 25 units, and repeat step E4. If no such users exist, but QUEUE [FLOOR] 
is not empty, send the front person of that queue immediately to step U5 
instead of U4, wait 25 units, and repeat step E4. But if QUEUE [FLOOR] 
is empty, set D1 i— 0, make D3 nonzero, and wait for some other activity 
to initiate further action. (Step E5 will send us to E6, or step U2 will 
restart E4.) 

E5. [Close doors.] If D1 ^ 0, wait 40 units and repeat this step (the doors flutter 
a little, but they spring open again, since someone is still getting out or in). 
Otherwise set D3 <— 0 and set the elevator to start at step E6 after 20 units 
of time. (This simulates closing the doors after people have finished getting 
in or out; but if a new user enters on this floor while the doors are closing, 
they will open again as stated in step U2.) 

E6. [Prepare to move.] Set CALLCAR [FLOOR] to zero; also set CALLUP [FLOOR] 
to zero if STATE ^ GOINGDOWN, and also set CALLD0WN [FLOOR] to zero if 
STATE ^ G0INGUP. (Note: If STATE = G0INGUP, the elevator does not clear 
out CALLD0WN, since it assumes that people who are going down will not 
have entered; but see exercise 6.) Now perform the DECISION subroutine. 

If STATE = NEUTRAL even after the DECISION subroutine has acted, go 
to El. Otherwise, if D2 ^ 0, cancel the elevator activity E9. Finally, if 
STATE = G0INGUP, wait 15 units of time (for the elevator to build up speed) 
and go to E7; if STATE = GOINGDOWN, wait 15 units and go to E8. 

E7. [Go up a floor.] Set FLOOR f- FLOOR + 1 and wait 51 units of time. If 
now CALLCAR [FLOOR] = 1 or CALLUP [FLOOR] = 1, or if ((FLOOR = 2 or 
CALLD0WN [FLOOR] = 1) and CALLUP [j] = CALLD0WN [j] = CALLCAR [j] = 0 

for all j > FLOOR), wait 14 units (for deceleration) and go to E2. Otherwise, 
repeat this step. 

E8. [Go down a floor.] This step is like E7 with directions reversed, and also 
the times 51 and 14 are changed to 61 and 23, respectively. (It takes the 
elevator longer to go down than up.) 

E9. [Set inaction indicator.] Set D2 <— 0 and perform the DECISION subroutine. 
(This independent action is initiated in step E3 but it is almost always 
canceled in step E6. See exercise 4.) | 


Table 1 

SOME ACTIONS OF THE ELEVATOR SYSTEM 
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Subroutine D (DECISION subroutine). This subroutine is performed at certain 
critical times, as specified in the coroutines above, when a decision about the 
elevator’s next direction is to be made. 

Dl. [Decision necessary?] If STATE ^ NEUTRAL, exit from this subroutine. 

D2. [Should doors open?] If the elevator is positioned at El and if CALLUP [2] , 
CALLCAR[2], and CALLD0WN[2] are not all zero, cause the elevator to start 
its activity E3 after 20 units of time, and exit from this subroutine. (If 
the DECISION subroutine is currently being invoked by the independent 
activity E9, it is possible for the elevator coroutine to be positioned at El.) 

D3. [Any calls?] Find the smallest j / FLOOR for which CALLUP [ j] , CALLCAR [ j] , 
or CALLDOWN [ j] is nonzero, and go on to step D4. But if no such j exists, 
then set j 2 if the DECISION subroutine is currently being invoked by 
step E6; otherwise exit from this subroutine. 

D4. [Set STATE.] If FLOOR > j, set STATE <- G0INGD0WN; if FLOOR < j, set 
STATE <- GOINGUP. 

D5. [Elevator dormant?] If the elevator coroutine is positioned at step El, and 
if j / 2, set the elevator to perform step E6 after 20 units of time. Exit 
from the subroutine. | 

The elevator system described above is quite complicated by comparison 
with other algorithms we have seen in this book, but the choice of a real-life 
system is more typical of a simulation problem than any cooked-up “textbook 
example” would ever be. 

To help understand the system, consider Table 1, which gives part of the 
history of one simulation. It is perhaps best to start by examining the simple 
case starting at time 4257: The elevator is sitting idly at floor 2 with its doors 
shut, when a user arrives (time 4384); let’s say the user’s name is Don. Two 
seconds later, the doors open, and Don gets in after two more seconds. By 
pushing button “3” he starts the elevator moving up; ultimately he gets off at 
floor 3 and the elevator returns to floor 2. 

The first entries in Table 1 show a much more dramatic scenario: A user 
calls the elevator to floor 0, but loses patience and gives up after 15.2 sec. The 
elevator stops at floor 0 but finds nobody there; then it heads to floor 4, since 
there are several calls wanting to go downward; etc. 

The programming of this system for a computer (in our case, MIX) merits 
careful study. At any given time during the simulation, we may have many 
simulated users in the system (in various queues and ready to “give up” at various 
times), and there is also the possibility of essentially simultaneous execution of 
steps E4, E5, and E9 if many people are trying to get out as the elevator is 
trying to close its doors. The passing of simulated time and the handling of 
“simultaneity” may be programmed by having each entity represented by a node 
that includes a NEXTTIME field (denoting the time when the next action for this 
entity is to take place) and a NEXTINST field (denoting the memory address where 
this entity is to start executing instructions, analogous to ordinary coroutine 
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QUEUE [1] 



Fig. 12. Some lists used in the elevator simulation program. (List heads appear at 
the left.) 


linkage). Each entity waiting for time to pass is placed in a doubly linked list 
called the WAIT list; this “agenda” is sorted on the NEXTTIME fields of its nodes, so 
that the actions may be processed in the correct sequence of simulated times. The 
program also uses doubly linked lists for the ELEVATOR and for the QUEUE lists. 

Each node representing an activity (whether a user or an elevator action) 
has the form 


+ 

IN 

LLINK1 

RLINK1 

+ 


NEXTTIM 

E 

+ 

NEXT 

INST 0 

0 39 

+ 

OUT 

LLINK2 

RLINK2 


( 6 ) 


Here LLINK1 and RLINK1 are the links for the WAIT list; LLINK2 and RLINK2 are 
used as links in the QUEUE lists or the ELEVATOR. The latter two fields and the 
IN and OUT field are relevant when node (6) represents a user, but they are not 
relevant for nodes that represent elevator actions. The third word of the node is 
actually a MIX “JMP” instruction. 

Figure 12 shows typical contents of the WAIT list, ELEVATOR list, and one of 
the QUEUE lists; each node in the QUEUE list is simultaneously in the WAIT list 
with NEXTINST = U4, but this has not been indicated in the figure, since the 
complexity of the linking would obscure the basic idea. 

Now let us consider the program itself. It is quite long, although (as with all 
long programs) it divides into small parts each of which is quite simple in itself. 
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First comes a number of lines of code that just serve to define the initial contents 
of the tables. There are several points of interest here: We have list heads for the 
WAIT list (lines 010-011), the QUEUE lists (lines 026-031), and the ELEVATOR list 
(lines 032-033). Each of them is a node of the form (6), but with unimportant 
words deleted; the WAIT list head contains only the first two words of a node, 
and the QUEUE and ELEVATOR list heads require only the last word of a node. 
We also have four nodes that are always present in the system (lines 012-023): 
USER1, a node that is always positioned at step U 1 ready to enter a new user 
into the system; ELEV1, a node that governs the main actions of the elevator 
at steps El, E2, E3, E4, E6, E7, and E8; and ELEV2 and ELEV3, nodes that 
are used for the elevator actions E5 and E9, which take place independently of 
other elevator actions with respect to simulated time. Each of these four nodes 
contains only three words, since they never appear in the QUEUE or ELEVATOR 
lists. The nodes representing each actual user in the system will appear in a 
storage pool following the main program. 


001 

* THE ELEVATOR SIMULATION 


002 

IN 

EQU 

1:1 

Definition of fields 

003 

LLINK1 

EQU 

2:3 

within nodes 

004 

RLINK1 

EQU 

4:5 


005 

NEXTINST 

EQU 

0:2 


006 

OUT 

EQU 

1:1 


007 

LLINK2 

EQU 

2:3 


008 

RLINK2 

EQU 

4:5 


009 

* FIXED- 

SIZE 

TABLES AND LIST HEADS 


010 

WAIT 

CON 

*+2(LLINKl) ,*+2(RLINKl) 

List head for WAIT list 

Oil 


CON 

0 

NEXTTIME = 0 always 

012 

USER1 

CON 

*-2(LLINKl) ,*-2(RLINKl) 

This node represents action 

013 


CON 

0 

U1 and it is initially the 

014 


JMP 

U1 

sole entry in the WAIT list. 

015 

ELEV1 

CON 

0 

This node represents the 

016 


CON 

0 

elevator actions, except 

on 


JMP 

El 

for E5 and E9. 

018 

ELEV2 

CON 

0 

This node represents the 

019 


CON 

0 

independent elevator 

020 


JMP 

E5 

action at E5. 

021 

ELEV3 

CON 

0 

This node represents the 

022 


CON 

0 

independent elevator 

023 


JMP 

E9 

action at E9. 

024 

AVAIL 

CON 

0 

Link to available nodes 

025 

TIME 

CON 

0 

Current simulated time 

026 

QUEUE 

EQU 

*-3 


027 


CON 

*-3(LLINK2) , *-3(RLINK2) 

List head for QUEUE [0] 

028 


CON 

*-3(LLINK2) ,*-3(RLINK2) 

List head for QUEUE [1] 

029 


CON 

*-3(LLINK2) ,*-3(RLINK2) 

All queues initially 

030 


CON 

*-3(LLINK2) ,*-3(RLINK2) 

are empty 

031 


CON 

*-3(LLINK2) ,*-3(RLINK2) 

List head for QUEUE [4] 
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032 

ELEVATOR 

EQU 

*-3 



033 


CON 

*-3(LLINK2) 

-3CRLINK2) List head for ELEVATOR 

034 


CON 

0 

> 


035 


CON 

0 


> “Padding” for CALL table 

036 


CON 

0 


(see lines 183-186) 

037 


CON 

0 


038 

CALL 

CON 

0 

CALLUP [0], CALLCARfO] , CALLDOWNfO] 

039 


CON 

0 

CALLUP [1], CALLCARfl] , CALLDOWN [1] 

040 


CON 

0 

CALLUP [2], CALLCAR[2] , CALLDOWN [2] 

041 


CON 

0 

CALLUP [3] , C ALLCAR [3] , CALLDOWN [3] 

042 


CON 

0 

CALLUP [4], CALLCAR[4] , CALLDOWN [4] 

043 


CON 

0 

] 


044 


CON 

0 


( “Padding” for CALL table 

045 


CON 

0 


(see lines 178-181) 

046 


CON 

0 


047 

D1 

CON 

0 

Indicates doors open, activity 

048 

D2 

CON 

0 

Indicates no prolonged standstill 

049 

D3 

CON 

0 

Indicates doors open, inactivity | 


The next part of the program coding contains basic subroutines and the 
main control routines for the simulation process. Subroutines INSERT and DELETE 
perform typical manipulations on doubly linked lists; they put the current node 
into or take it out of a QUEUE or ELEVATOR list. (In the program, the “current 
node C is always represented by index register 6.) There are also subroutines for 
the WAIT list: Subroutine SORTIN adds the current node to the WAIT list, sorting 
it into the right place based on its NEXTTIME field. Subroutine IMMED inserts the 
current node at the front of the WAIT list. Subroutine HOLD puts the current node 
into the WAIT list, with NEXTTIME equal to the current time plus the amount in 
register A. Subroutine DELETEW deletes the current node from the WAIT list. 

The routine CYCLE is the heart of the simulation control: It decides which 
activity is to act next (namely, the first element of the WAIT list, which we know 
is nonempty), and jumps to it. There are two special entrances to CYCLE: CYCLE1 
first sets NEXTINST in the current node, and HOLDC is the same with an additional 
call on the HOLD subroutine. Thus, the effect of the instruction “JMP HOLDC” with 
amount t in register A is to suspend activity for t units of simulated time and 
then to return to the following location. 


050 

* SUBROUTINES AND CONTROL 

051 

INSERT 

STJ 

9F 

052 


LD2 

3,1(LLINK2) 

053 


ST2 

3,6(LLINK2) 

054 


ST6 

3, 1 (LLINK2) 

055 


ST6 

3,2(RLINK2) 

056 


ST1 

3,6(RLINK2) 

057 

9H 

JMP 

* 

058 

DELETE 

STJ 

9F 

059 


LD1 

3,6(LLINK2) 

060 


LD2 

3,6(RLINK2) 


ROUTINE 

Insert NODE(C) to left of NODE(rll): 
rI2 <- LLINK2 (rll) . 

LLINK2CC) «- rI2. 

LLINK2(rIl) <- C. 

RLINK2 (rI2) <- C. 

RLINK2(C) <- rll. 

Exit from subroutine. 

Delete NODE(C) from its list: 

P «- LLINK2CC). 

Q <- RLINK2CC). 
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061 


ST1 

3,2(LLINK2) 

LLINK2CQ) <- P. 

062 


ST2 

3, 1 (RLINK2) 

RLINK2 (P) <- Q. 

063 

9H 

JMP 

* 

Exit from subroutine. 

064 

IMMED 

STJ 

9F 

Insert N0DE(C) first in WAIT list: 

065 


LDA 

TIME 


066 


STA 

1,6 

Set NEXTTIME (C) «- TIME. 

067 


ENT1 

WAIT 

P <- LOC(WAIT) . 

068 


JMP 

2F 

Insert N0DE(C) to right of N0DE(P) . 

069 

HOLD 

ADD 

TIME 

rA <- TIME + rA. 

070 

S0RTIN 

STJ 

9F 

Sort N0DE(C) into WAIT list: 

071 


STA 

1,6 

Set NEXTTIME (C) <- rA. 

072 


ENT1 

WAIT 

P LOC(WAIT) . 

073 


LD1 

0 , 1 (LLINK1) 

P «- LLINKl(P) . 

074 


CMPA 

1,1 

Compare NEXTTIME fields, right to left. 

075 


JL 

*-2 

Repeat until NEXTTIME (C) > NEXTTIME(P) 

076 

2H 

LD2 

0,1(RLINK1) 

Q <- RLINKl(P). 

077 


ST2 

0,6(RLINK1) 

RLINKl(C) <- Q. 

078 


ST1 

0,6(LLINK1) 

LLINKl(C) <- P. 

079 


ST6 

0, l(RLINKl) 

RLINKl(P) <- C. 

080 


ST6 

0 , 2 (LLINK1) 

LLINKl(Q) <- C. 

081 

9H 

JMP 

* 

Exit from subroutine. 

082 

DELETEW 

STJ 

9F 

Delete N0DE(C) from WAIT list: 

083 


LD1 

0,6(LLINK1) 

(This is same as lines 058-063 

084 


LD2 

0,6(RLINK1) 

except LLINK1, RLINK1 are used 

085 


ST1 

0,2(LLINK1) 

instead of LLINK2, RLINK2.) 

086 


ST2 

0 , 1 (RLINK1) 


087 

9H 

JMP 

* 


088 

CYCLE1 

STJ 

2, 6 (NEXTINST) 

Set NEXTINST (C) <- rj. 

089 


JMP 

CYCLE 


090 

H0LDC 

STJ 

2,6(NEXTINST) 

Set NEXTINST (C) <- rJ. 

091 


JMP 

HOLD 

Insert N0DE(C) in WAIT, delay rA. 

092 

CYCLE 

LD6 

WAIT(RLINKl) 

Set current node C <— RLINK1 (LOC(WAIT) ) 

093 


LDA 

1,6 


094 


STA 

TIME 

TIME «- NEXTTIME (C). 

095 


JMP 

DELETEW 

Remove N0DE(C) from WAIT list. 

096 


JMP 

2,6 

Jump to NEXTINST (C) . | 


Now comes the program for Coroutine U. At the beginning of step Ul, 
the current node C is USER1 (see lines 012-014 above), and lines 099-100 of the 
program cause USER1 to be reinserted into the WAIT list so that the next user 
will be generated after INTERTIME units of simulated time. The following lines 
101 114 take care of setting up a node for the newly generated user; the IN and 
OUT floors are recorded in this node position. The AVAIL stack is singly linked 
in the RLINK1 field of each node. Note that lines 101 108 perform the action 
“C AVAIL” using the P00LMAX technique, 2.2.3— (7) ; no test for OVERFLOW is 
necessary here, since the total size of the storage pool (the number of users in 
the system at any one time) rarely exceeds 10 nodes (40 words). The return of 
a node to the AVAIL stack appears in lines 156-158. 
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Throughout the program, index register 4 equals the variable FLOOR, and 
index register 5 is positive, negative, or zero, depending on whether STATE = 
GOINGUP, GOINGDOWN, or NEUTRAL. The variables CALLUP[j], CALLCAR[j], and 
CALLDOWN [j] occupy the respective fields (1:1), (3:3), and (5:5) of location 


CALL + j 




097 

* COROUTINE U 

Ul. Enter, prepare for successor. 

098 

U1 

JMP 

VALUES 

Set INFLOOR, OUTFLOOR, GIVEUPTIME, INTERTIME 

099 


LDA 

INTERTIME 

INTERTIME is computed by VALUES subroutine. 

100 


JMP 

HOLD 

Put NODE(C) in WAIT, delay INTERTIME. 

101 


LD6 

AVAIL 

C 4- AVAIL. 

102 


J6P 

IF 

If AVAIL / A, jump. 

103 


LD6 

P00LMAX(0:2) 


104 


INC6 

4 

C 4- POOLMAX + 4. 

105 


ST6 

POOLMAX (0 : 2) 

POOLMAX 4- C. 

106 


JMP 

*+3 

Assume that memory overflow won’t happen. 

107 

1H 

LDA 

0,6(RLINK1) 

108 


STA 

AVAIL 

AVAIL 4- RLINK1 (AVAIL) . 

109 


LD1 

INFLOOR 

rll 4— INFLOOR (computed by VALUES above). 

110 


ST1 

0,6(IN) 

IN(C) 4- rll. 

111 


LD2 

0UTFL00R 

rI2 4— OUTFLOOR (computed by VALUES). 

112 


ST2 

3, 6 (OUT) 

OUT(C) 4- rI2. 

113 


ENTA 

39 

Put constant 39 (JMP operation code) 

114 


STA 

2,6 

into third word of node format (6). 

115 

U2 

ENTA 

0,4 

U2. Signal and wait. Set rA 4— FT.nnR 

116 


DECA 

0,1 

FLOOR - IN. 

117 


ST6 

TEMP 

Save value of C. 

118 


JANZ 

2F 

Jump if FLOOR ^ IN. 

119 


ENT6 

ELEV1 

Set C 4- LOC(ELEVl) . 

120 


LDA 

2 , 6 (NEXTINST) 

Is elevator positioned at E6? 

121 


DECA 

E6 


122 


JANZ 

3F 


123 


ENTA 

E3 

If so, reposition it at E3. 

124 


STA 

2, 6 (NEXTINST) 


125 


JMP 

DELETEW 

Remove it from WAIT list 

126 


JMP 

4F 

and reinsert it at front of WAIT. 

127 

3H 

LDA 

D3 


128 


JAZ 

2F 

Jump if D3 = 0. 

129 


ST6 

D1 

Otherwise make D1 nonzero. 

130 


STZ 

D3 

Set D3 4- 0. 

131 

4H 

JMP 

IMMED 

Insert ELEV1 at front of WAIT list. 

132 


JMP 

U3 

(rll and rI2 have changed.) 

133 

2H 

DEC2 

0,1 

rI2 4- OUT - IN. 

134 


ENTA 

1 


135 


J2P 

*+3 

Jump if going up. 

136 


STA 

CALL, 1(5: 5) 

Set CALLDOWN [IN] 4- 1. 

137 


JMP 

*+2 


138 


STA 

CALL, 1(1:1) 

Set CALLUP [IN] 4- 1. 

139 


LDA 

D2 
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140 


JAZ 

*+3 

141 


LDA 

ELEV1+2 (NEXTINST) 

142 


DECA 

El 

143 


JAZ 

DECISION 

144 

U3 

LD6 

TEMP 

145 


LD1 

0,6(IN) 

146 


ENT1 

QUEUE, 1 

147 


JMP 

INSERT 

148 

U4A 

LDA 

GIVEUPTIME 

149 


JMP 

HOLDC 

150 

U4 

LDA 

0,6(IN) 

151 


DECA 

0,4 

152 


JANZ 

*+3 

153 


LDA 

D1 

154 


JANZ 

U4A 

155 

U6 

JMP 

DELETE 

156 


LDA 

AVAIL 

157 


STA 

0 , 6 (RLINK1) 

158 


ST6 

AVAIL 

159 


JMP 

CYCLE 

160 

U5 

JMP 

DELETE 

161 


ENT1 

ELEVATOR 

162 


JMP 

INSERT 

163 


ENTA 

1 

164 


LD2 

3, 6 (OUT) 

165 


STA 

CALL, 2 (3: 3) 

166 


J5NZ 

CYCLE 

167 


DEC2 

0,4 

168 


ENT5 

0,2 

169 


ENT6 

ELEV2 

170 


JMP 

DELETEW 

171 


ENTA 

25 

172 


JMP 

E5A 


If D2 = 0, call the DECISION subroutine. 

If the elevator is at El, call 
the DECISION subroutine. 

U3. Enter queue. 

rll 4- LOC (QUEUE [IN] ) . 

Insert NODE(C) at right end of QUEUE[IN] 

Wait GIVEUPTIME units. 

U4. Give u p. 

IN (C) - FLOOR. 

FLOOR = IN (C) . 

See exercise 7. 

U6. Get out. NODE(C) is deleted 
from QUEUE or ELEVATOR. 

AVAIL 4= C. 

Continue simulation. 

U5. Get in. NODE(C) is deleted 
from QUEUE. 

Insert it at right of ELEVATOR. 


Set CALLCAR [OUT (C) ] 4- 1. 

Jump if STATE ^ NEUTRAL. 
rI2 4- OUT(C) - FLOOR. 

Set STATE to proper direction. 

Set C 4- L0C(ELEV2) . 

Remove E5 action from WAIT list. 

Restart E5 action 25 units from now. | 


The program for Coroutine E is a rather straightforward rendition of the 
semiformal description given earlier. Perhaps the most interesting portion is the 
preparation for the elevator’s independent actions in step E3, and the searching 
of the ELEVATOR and QUEUE lists in step E4. 


173 

* COROUTINE E 


174 

E1A 

JMP 

CYCLE 1 

Set NEXTINST 4- El, go to CYCLE. 

175 

El 

EQU 

* 

El. Wait for call, (no action) 

176 

E2A 

JMP 

HOLDC 


177 

E2 

J5N 

IF 

E2. Change of state? 

178 


LDA 

CALL+1,4 

State is GOINGUP. 

179 


ADD 

CALL+2,4 


180 


ADD 

CALL+3,4 


181 


ADD 

CALL+4,4 


182 


JAP 

E3 

Are there calls for higher floors? 
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183 

LDA 

CALL-1, 4(3:3) 

If not, have passengers in the 

184 

ADD 

CALL-2, 4 (3: 3) 

elevator called for lower floors? 

185 

ADD 

CALL-3, 4 (3: 3) 


186 

ADD 

CALL-4, 4(3:3) 


187 

JMP 

2F 


188 

1H LDA 

CALL- 1,4 

State is GOINGDOWN. 

189 

ADD 

CALL-2,4 

Actions are like lines 178-186. 


196 


ADD 

CALL+4,4(3:3) 


197 

2H 

ENN5 

0,5 

Reverse direction of STATE. 

198 


STZ 

CALL, 4 

Set CALL variables to zero. 

199 


JANZ 

E3 

Jump if called to the opposite direction; 

200 


ENT5 

0 

otherwise set STATE <— NEUTRAL. 

201 

E3 

ENT6 

ELEV3 

E3. Open doors. 

202 


LDA 

0,6 

If activity E9 is already scheduled, 

203 


JANZ 

DELETEW 

remove it from the WAIT list. 

204 


ENTA 

300 


205 


JMP 

HOLD 

Schedule activity E9 after 300 units. 

206 


ENT6 

ELEV2 


207 


ENTA 

76 


208 


JMP 

HOLD 

Schedule activity E5 after 76 units. 

209 


ST6 

D2 

Set D2 nonzero. 

210 


ST6 

D1 

Set D1 nonzero. 

211 


ENTA 

20 


212 

E4A 

ENT6 

ELEV1 


213 


JMP 

HOLDC 


214 

E4 

ENTA 

0,4 

E4. Let people out. in. 

215 


SLA 

4 

Set OUT field of rA to FLOOR. 

216 


ENT6 

ELEVATOR 

C <- LOC (ELEVATOR). 

217 

1H 

LD6 

3,6(LLINK2) 

C <— LLINK2(C) . 

218 


CMP6 

=ELEVAT0R= 

Search ELEVATOR list, right to left. 

219 


JE 

IF 

If C = LOC (ELEVATOR) , search is complete. 

220 


CMPA 

3, 6 (OUT) 

Compare OUT(C) with FLOOR. 

221 


JNE 

IB 

If not equal, continue searching; 

222 


ENTA 

U6 

otherwise prepare to send user to U6. 

223 


JMP 

2F 


224 

1H 

LD6 

QUEUE+3,4(RLINK2) 

Set C <- RLINK2 (LOC (QUEUE [FLOOR] ) ) . 

225 


CMP6 

3,6(RLINK2) 

Is C = RLINK2 (C) ? 

226 


JE 

IF 

If so, the queue is empty. 

227 


JMP 

DELETEW 

If not, cancel action U4 for this user. 

228 


ENTA 

U5 

Prepare to replace U4 by U5. 

229 

2H 

STA 

2 ,6 (NEXTINST) 

Set NEXTINST (C). 

230 


JMP 

IMMED 

Put user at the front of the WAIT list. 

231 


ENTA 

25 


232 


JMP 

E4A 

Wait 25 units and repeat E4. 

233 

1H 

STZ 

D1 

Set D1 0. 

234 


ST6 

D3 

Set D3 nonzero. 

235 


JMP 

CYCLE 

Return to simulate other events. 
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236 

E5A 

JMP 

HOLDC 


237 

E5 

LDA 

D1 

E5. Close doors. 

238 


JAZ 

*+3 

Is D1 = 0? 

239 


ENTA 

40 

If not, people are still getting in or out. 

240 


JMP 

E5A 

Wait 40 units, repeat E5. 

241 


STZ 

D3 

If D1 = 0, set D3 <- 0. 

242 


ENT6 

ELEV1 


243 


ENTA 

20 


244 


JMP 

HOLDC 

Wait 20 units, then go to E6. 

245 

E6 

J5N 

*+2 

E6. Prepare to move. 

246 


STZ 

CALL, 4(1:3) 

If STATE / GOINGDOWN, CALLUP and CALLCAR 

247 


J5P 

*+2 

on this floor are reset. 

248 


STZ 

CALL, 4(3:5) 

If / GOINGUP, reset CALLCAR and CALLDOWN. 

249 


J5Z 

DECISION 

Perform DECISION subroutine. 

250 

E6B 

J5Z 

E1A 

If STATE = NEUTRAL, go to El and wait. 

251 


LDA 

D2 


252 


JAZ 

*+4 


253 


ENT6 

ELEV3 

Otherwise, if D2 ^ 0, 

254 


JMP 

DELETEW 

cancel activity E9 

255 


STZ 

ELEV3 

(see line 202). 

256 


ENT6 

ELEV1 


257 


ENTA 

15 

Wait 15 units of time. 

258 


J5N 

E8A 

If STATE = GOINGDOWN, go to E8. 

259 

E7A 

JMP 

HOLDC 


260 

E7 

INC4 

1 

E7. Go up a floor. 

261 


ENTA 

51 


262 


JMP 

HOLDC 

Wait 51 units. 

263 


LDA 

CALL, 4(1:3) 

Is CALLCAR [FLOOR] or CALLUP [FLOOR] 0? 

264 


JAP 

IF 


265 


ENT1 

-2,4 

If not, 

266 


J1Z 

2F 

is FLOOR = 2? 

267 


LDA 

CALL, 4(5:5) 

If not, is CALLDOWN [FLOOR] / 0? 

268 


JAZ 

E7 

If not, repeat step E7. 

269 

2H 

LDA 

CALL+1,4 


270 


ADD 

CALL+2,4 


271 


ADD 

CALL+3,4 


272 


ADD 

CALL+4,4 


273 


JANZ 

E7 

Are there calls for higher floors? 

274 

1H 

ENTA 

14 

It is time to stop the elevator. 

275 


JMP 

E2A 

Wait 14 units and go to E2. 

276 

E8A 

JMP 

HOLDC 






(See exercise 8.) 
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JMP 

E2A 


293 

E9 

STZ 

0,6 

E9. Set inaction indicator. (See line 202.) 

294 


STZ 

D2 

D2 <- 0. 

295 


JMP 

DECISION 

Perform DECISION subroutine. 

296 


JMP 

CYCLE 

Return to simulation of other events. | 
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We will not consider here the DECISION subroutine (see exercise 9), nor the 
VALUES subroutine that is used to specify the demands on the elevator. At the 
very end of the program comes the code 


BEGIN ENT4 2 
ENT5 0 
JMP CYCLE 
POOLMAX NOP POOL 
POOL END BEGIN 


Start with FLOOR = 2 

and STATE = NEUTRAL. 

Begin simulation. 

Storage pool follows literals, temp storage. | 


The program above does a fine job of simulating the elevator system, as it 
goes through its paces. But it would be useless to run this program, since there 
is no output! Actually, the author added a PRINT subroutine that was called at 
most of the critical steps in the program above, and this was used to prepare 
Table 1; the details have been omitted, since they are very straightforward but 
they only clutter up the code. 

Several programming languages have been devised that make it quite easy 
to specify the actions in a discrete simulation, and to use a compiler to translate 
these specifications into machine language. Assembly language was used in this 
section, of course, since we are concerned here with the basic techniques of linked 
list manipulation, and we want to see the details of how discrete simulations can 
actually be performed by a computer that has a one-track mind. The technique 
of using a WAIT list or agenda to control the sequencing of coroutines, as we have 
done in this section, is called quasiparallel processing. 

It is quite difficult to give a precise analysis of the running time of such a long 
program, because of the complex interactions involved; but large programs often 
spend most of their time in comparatively short routines doing comparatively 
simple things. Therefore we can usually get a good indication of the overall 
efficiency by using a special trace routine called a profiler, which executes the 
program and records how often each instruction is performed. This identifies the 
“bottlenecks,” the places that should be given special attention. [See exercise 
1.4. 3. 2-7. See also Software Practice <fc Experience 1 (1971), 105 133. for 
examples of such studies on randomly selected FORTRAN programs found in 
wastebaskets at the Stanford Computer Center.] The author made such an 
experiment with the elevator program above, running it for 10000 units of 
simulated elevator time; 26 users entered the simulated system. The instructions 
in the S0RTIN loop, lines 073-075, were executed by far the most often, 1432 
times, while the S0RTIN subroutine itself was called 437 times. The CYCLE 
routine was performed 407 times; so we could gain a little speed by not calling 
the DELETEW subroutine at line 095: The four lines of that subroutine could be 
written out in full (to save 4 u each time CYCLE is used). The profiler also showed 
that the DECISION subroutine was called only 32 times and the loop in E4 (lines 
217-219) was executed only 142 times. 

It is hoped that some reader will learn as much about simulation from the 
example above as the author learned about elevators while the example was 
being prepared. 
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EXERCISES 

1. [21] Give specifications for the insertion and deletion of information at the left 
end of a doubly linked list represented as in (i). (With the dual operations at the right 
end, which are obtained by symmetry, we therefore have all the actions of a general 
deque.) 

► 2. [22] Explain why a list that is singly linked cannot allow efficient operation as a 
general deque; the deletion of items can be done efficiently at only one end of a singly 
linked list. 

► 3. [22] The elevator system described in the text uses three call variables, CALLUP, 
CALLCAR, and CALLDOWN, for each floor, representing buttons that have been pushed by 
the users in the system. It is conceivable that the elevator actually needs only one or 
two binary variables for the call buttons on each floor, instead of three. Explain how 
an experimenter could push buttons in a certain sequence with this elevator system to 
prove that there are three independent binary variables for each floor (except the top 
and bottom floors). 

4. [24] Activity E9 in the elevator coroutine is usually canceled by step E6; and 
even when it hasn’t been canceled, it doesn’t do very much. Explain under what 
circumstances the elevator would behave differently if activity E9 were deleted from 
the system. Would it, for example, sometimes visit floors in a different order? 

5. [20] In Table 1, user 10 arrived on floor 0 at time 1048. Show that if user 10 had 
arrived on floor 2 instead of floor 0, the elevator would have gone up after receiving 
its passengers on floor 1, instead of down, in spite of the fact that user 8 wants to go 
down to floor 0. 

6. [23] During the time period 1183-1233 in Table 1, users 7, 8, and 9 all get in the 
elevator on floor 1; then the elevator goes down to floor 0 and only user 8 gets out. 
Now the elevator stops again on floor 1, presumably to pick up users 7 and 9 who are 
already aboard; nobody is actually on floor 1 waiting to get in. (This situation occurs 
not infrequently at Caltech; if you get on the elevator going the wrong way, you must 
wait for an extra stop as you go by your original floor again.) In many elevator systems, 
users 7 and 9 would not have boarded the elevator at time 1183, since lights outside 
the elevator would show that it was going down, not up; those users would have waited 
until the elevator came back up and stopped for them. On the system described, there 
are no such lights and it is impossible to tell which way the elevator is going to go until 
you are in it; hence Table 1 reflects the actual situation. 

What changes should be made to coroutines U and E if we were to simulate the 
same elevator system, but with indicator lights, so that people do not get on the elevator 
when its state is contrary to their desired direction? 

7. [25] Although bugs in programs are often embarrassing to a programmer, if we 
are to learn from our mistakes we should record them and tell other people about them 
instead of forgetting them. The following error (among others) was made by the author 
when he first wrote the program in this section: Line 154 said “JANZ CYCLE” instead 
of ‘'JANZ U4A” . The reasoning was that if indeed the elevator had arrived at this user’s 
floor, there was no need to perform the “give up” activity U4 any more, so we could 
simply go to CYCLE and continue simulating other activities. What was the error? 

8. [21] Write the code for step E8, lines 277-292, which has been omitted from the 
program in the text. 


298 INFORMATION STRUCTURES 


2.2.5 


9. [23] Write the code for the DECISION subroutine, which has been omitted from 
the program in the text. 

10. [40] It is perhaps significant to note that although the author had used the 
elevator system for years and thought he knew it well, it wasn’t until he attempted 
to write this section that he realized there were quite a few facts about the elevator’s 
system of choosing directions that he did not know. He went back to experiment with 
the elevator six separate times, each time believing he had finally achieved a complete 
understanding of its modus operandi. (Now he is reluctant to ride it for fear that some 
new facet of its operation will appear, contradicting the algorithms given.) We often 
fail to realize how little we know about a thing until we attempt to simulate it on a 
computer. 

Try to specify the actions of some elevator you are familiar with. Check the 
algorithm by experiments with the elevator itself (looking at its circuitry is not fair!); 
then design a discrete simulator for the system and run it on a computer. 

►11. [21] [A spar se-update memory.) The following problem often arises in synchro- 
nous simulations: The system has n variables V[l ] , , V[n] , and at every simulated 

step new values for some of them are calculated from the old values. These calculations 
are assumed done “simultaneously” in the sense that the variables do not change to 
their new values until after all assignments have been made. Thus, the two statements 

V [1] <— V[2] and V[2] <- V [1] 

appearing at the same simulated time would interchange the values of V [1] and V [2] ; 
this is quite different from what would happen in a sequential calculation. 

The desired action can of course be simulated by keeping an additional table 
NEWV [1] , ..., NEWV [n] . Before each simulated step, we could set NEWV [A] «- v [fc] for 
1 < A: < n, then record all changes of V[fc] in NEWV [A], and finally, after the step 
we could set V[A] t— NEWV [A], 1 < A < n. But this “brute force” approach is not 
completely satisfactory, for the following reasons: (1) Often n is very large, but the 
number of variables changed per step is rather small. (2) The variables are often not 
arranged in a nice table V [1] , . . . , V [n] , but are scattered throughout memory in a 
rather chaotic fashion. (3) This method does not detect the situation (usually an error 
in the model) when one variable is given two values in the same simulated step. 

Assuming that the number of variables changed per step is rather small, design 
an efficient algorithm that simulates the desired actions, using two auxiliary tables 
NEWV [A] and LINK [A] , 1 < A < n. If possible, your algorithm should give an error stop 
if the same variable is being given two different values in the same step. 

► 12 . [22] Why is it a good idea to use doubly linked lists instead of singly linked or 
sequential lists in the simulation program of this section? 

2.2.6. Arrays and Orthogonal Lists 

One of the simplest generalizations of a linear list is a two-dimensional or higher- 
dimensional array of information. For example, consider the case of an m x n 
matrix 


/ A [1 , 1] 

A [1 ,2] . 

A [1 , n] 

A [2 , 1] 

A [2, 2] . 

A[2,n] 

^ A [to , 1] 

A [to , 2] 

. . A [to , n] 
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In this two-dimensional array, each node A [j , h] belongs to two linear lists: 
the “row j" list A[j,l], A[j,2], ..., A [ j , n] and the “column k ” list A[1,A;], 
A [2 , A-] , . . . , A [m , A:] . These orthogonal row and column lists essentially account 
for the two-dimensional structure of a matrix. Similar remarks apply to higher- 
dimensional arrays of information. 

Sequential Allocation. When an array like (i) is stored in sequential memory 
locations, storage is usually allocated so that 

L0C(A[J,K] ) = no T ui-J T u 2 K, ( 2 ) 

where ao , 01 , and 02 are constants. Let us consider a more general case: 
Suppose we have a four-dimensional array with one- word elements Q[I,J,K,L] 
for 0 < I < 2, 0 < J < 4, 0 < K < 10, 0 < L < 2. We would like to allocate 
storage so that 

LOC (Q [I , J , K , L] ) = ao + eql + 02 J + 0 S 3 K + (Z 4 L. ( 3 ) 

This means that a change in I, J, K, or L leads to a readily calculated change in 
the location of Q[I, J,K,L] . The most natural (and most commonly used) way 
to allocate storage is to arrange the array elements according to the lexicographic 
order of their indices (exercise 1.2.1-15(d)) , sometimes called “row major order”: 

Q[ 0 , 0 , 0 , 0 ], Q [0, 0 , 0 , 1 ], Q [ 0 , 0 , 0 , 2 ], Q[ 0 , 0 , 1 , 0 ], Q[ 0 , 0 , 1 , 1 ], ..., 

Q [0,0, 10,2] , Q[0, 1,0,0], ..., Q [0,4, 10,2] , Q [1, 0,0,0], ..., 

Q [2,4, 10, 2]. 

It is easy to see that this order satisfies the requirements of ( 3 ), and we have 

L0C(Q [I , J ,K ,L] ) = LOCCQfO.O.O.O]) + 1651 + 33J + 3K + L. ( 4 ) 

In general, given a A'-dimensional array with c-word elements A [1 1 , 12 , • • • , Ifc] 
for 

0 < Ii < di, 0 <I 2 <d 2 , 0 < I fc < d k , 

we can store it in memory as 
L0 C(A[Ii,I 2 , ... ,I fc ]) 

= LOC (A [0 ,0, . . . ,0] ) + C (d 2 + 1) ■ • ■ (dfc + 1) Ii + • • • + c (d^ + l)lfc_i + clfc 
= LOC ( A [0 , 0 , . . . , 0] ) + arlr ’ ( 5 ) 

l<r</c 

where 

a r — c (d s + 1 ). ( 6 ) 

r<s<k 

To see why this formula works, observe that a r is the amount of memory needed 
to store the subarray A [I x , ... ,I r , J r+1 , . . . , J*.] if I 1? . . . , I r are constant and 
J r+ i, . . . , Jfc vary through all values 0 < J r+ i < d r+ 1 , • • ■ , 0 < J fc < dfc; hence 
by the nature of lexicographic order the address of A [I ! , ... , I*.] should change 
by precisely this amount when I r changes by 1 . 
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Formulas ( 5 ) and ( 6 ) correspond to the value of the number I : I 2 . . . I fc in a 
mixed-radix number system. For example, if we had the array TIME [W , D , H , M , S] 

with 0 < W < 4, 0 < D < 7, 0 < H < 24, 0 < M < 60, and 0 < S < 60, the 
location of TIME[W,D,H,M,S] would be the location of TIME [0,0, 0,0,0] plus 
the quantity “W weeks + D days + H hours + M minutes + S seconds” converted to 
seconds. Of course, it takes a pretty fancy application to make use of an array 
that has 2,419,200 elements. 

The normal method for storing arrays is generally suitable when the array 
has a complete rectangular structure, so that all elements A [lx , I 2 , ... , I fe ] are 
present for indices in the independent ranges h < Ii < ui, l 2 < I 2 < u 2 , 
h < I k < u k . Exercise 2 shows how to adapt ( 5 ) and ( 6 ) to the case when the 
lower bounds (l x , . . . , l k ) are not ( 0 , 0 , . . . , 0 ). 

But there are many situations in which an array is not perfectly rectangular. 
Most common is the triangular matrix, where we want to store only the entries 
A [j , fc] for, say, 0 < k < j < n: 


/ A [0 , 0] \ 

A [1,0] A [1 , 1] 

\ A [n , 0] A[n,l] ... A[n,n]/ 


(7) 


We may know that all other entries are zero, or that A [ 7 , /,;] = A [fc,j] , so that 
only half of the values need to be stored. If we want to store the lower triangular 
matrix ( 7 ) in ~{n + l)(n + 2 ) consecutive memory positions, we are forced to 
give up the possibility of linear allocation as in Eq. ( 2 ), but we can ask instead 
for an allocation arrangement of the form 


L0C (A [ J , K] ) =a 0 + / 1 (J) +/ 2 (K) (8) 

where /1 and / 2 are functions of one variable. (The constant a Q may be absorbed 
into either f x or / 2 if desired.) When the addressing has the form ( 8 ), a random 
element A [j , fc] can be quickly accessed if we keep two (rather short) auxiliary 
tables of the values of f x and / 2 ; therefore these functions need to be calculated 
only once. 

It turns out that lexicographic order of indices for the array ( 7 ) satisfies 
condition ( 8 ), and with one- word entries we have in fact the simple formula 

L0C(A[J,K] ) = L0C(A[0,0] ) + J(J + ^ + K . ( 9 ) 

But there is actually a far better way to store triangular matrices, if we are 
fortunate enough to have two of them with the same size. Suppose that we want 
to store both A [ j , fc] and B [j , fc] for 0 < k < j < n. Then we can fit them both 
into a single matrix C[j,fc] for 0 < j < n, 0 < k < n + 1, using the convention 


A [ j ,ki] ~ C [ j , k ] 


B[j,fc] = C lk,j + 1] . 


( 10 ) 
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Thus 


/C[0,0] C[0,1] C [0 , 2] . 
C[1,0] C[l,l] C [1 , 2] . 

. . C [0 , n + 1] 

. . C [1 ,n+l] 


/ A [0,0] B [0 , 0] B [1 , 0] . 
A [1 , 0] A [1 , 1] B [1 , 1] . 

• • B in , 0] \ 

. . B [n , 1] 

\ C [n , 0] C [n , 1] C [n , 2] . 

• . C[n,n+1] / 


\A[n,0] A[n,l] A[n,2] . 

• • B in, n] / 


The two triangular matrices are packed together tightly within the space of 
(n + l)(n + 2) locations, and we have linear addressing as in ( 2 ). 

The generalization of triangular matrices to higher dimensions is called a 
tetrahedral array. This interesting topic is the subject of exercises 6 through 8. 

As an example of typical programming techniques for use with sequentially 
stored arrays, see exercise 1.3.2-10 and the two answers given for that exercise. 
The fundamental techniques for efficient traversal of rows and columns, as well 
as the uses of sequential stacks, are of particular interest within those programs. 

Linked Allocation. Linked memory allocation also applies to higher-dimen- 
sional arrays of information in a natural way. In general, our nodes can contain 
k link fields, one for each list the node belongs to. The use of linked memory is 
generally for cases in which the arrays are not strictly rectangular in character. 



w 










w 





22hl 

Q 



w 

< 





w ^ w w 

S3 



w 

s 


CN 

CO 

CO 

Sown 

O 

Q 

w 

< 

w 

CN 

CN 

CN 

CN CN 

w w cti < 

W 

W 


s: 

W 

< 

< 

<C 

< < 


CQ 

W 

Q 



Female, age 21, brown eyes, dark hair 
Male, age 24, brown eyes, dark hair 
Female, age 22, green eyes, blonde hair 
Male, age 28, hazel eyes, blond hair 
Female, age 22, blue eyes, red hair 
Female, age 21, blue eyes, blonde hair 


Fig. 13. Each node in four different lists. 

As an example, we might have a list in which every node represents a person, 
with four link fields: SEX, AGE, EYES, and HAIR. In the EYES field we link together 
all nodes with the same eye color, etc. (See Fig. 13.) It is easy to visualize efficient 
algorithms for inserting new people into the list; deletion would, however, be 
much slower, unless we used double linking. We can also conceive of algorithms 
of varying degrees of efficiency for doing things like “Find all blue-eyed blonde 
women of ages 21 through 23”; see exercises 9 and 10. Problems in which each 
node of a list is to reside in several kinds of other lists at once arise rather 
frequently; indeed, the elevator system simulation described in the preceding 
section has nodes that are in both the QUEUE and WAIT lists simultaneously. 
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As a detailed example of the use of linked allocation for orthogonal lists, 
we will consider the case of sparse matrices (that is, matrices of large order in 
which most of the elements are zero). The goal is to operate on these matrices 
as though the entire matrix were present, but to save great amounts of time 
and space because the zero entries need not be represented. One way to do 
this, intended for random references to elements of the matrix, would be to use 
the storage and retrieval methods of Chapter 6, to find A[j,A] from the key 
“[), A]”; however, there is another way to deal with sparse matrices that is often 
preferable because it reflects the matrix structure more appropriately, and this 
is the method we will discuss here. 

The representation we will discuss consists of circularly linked lists for each 
row and column. Every node of the matrix contains three words and five fields: 


(n) 


Here ROW and COL are the row and column indices of the node; VAL is the value 
stored at that part of the matrix; LEFT and UP are links to the next nonzero 
entry to the left in the row, or upward in the column, respectively. There are 
special list head nodes, BASER0W[i] and BASEC0L[j] , for every row and column. 
These nodes are identified by 

C0L(L0C(BASER0W[i] )) <0 and R0W(L0C(BASEC0L [j] ) ) < 0. 

As usual in a circular list, the LEFT link in BASER0W[i] is the location of the 
rightmost value in that row, and UP in BASECOL [ j] points to the bottom-most 
value in that column. For example, the matrix 

/ 50 0 0 0 

10 0 20 0 

0 0 0 0 

\ —30 0 -60 5 

would be represented as shown in Fig. 14. 

Using sequential allocation of storage, a 200 x 200 matrix would take 40000 
words, and this is more memory than many computers used to have; but a 
suitably sparse 200 x 200 matrix can be represented as above even in Mix’s 
4000- word memory. (See exercise 11.) The amount of time taken to access a 
random element A[j,A] is also quite reasonable, if there are but few elements 
in each row or column; and since most matrix algorithms proceed by walking 
sequentially through a matrix, instead of accessing elements at random, this 
linked representation often works faster than a sequential one. 

As a typical example of a nontrivial algorithm dealing with sparse matrices 
in this form, we will consider the pivot step operation, which is an important 
part of algorithms for solving linear equations, for inverting matrices, and for 
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Fig. 14. Representation of matrix ( 12 ), with nodes in the format 
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solving linear programming problems by the simplex method. A pivot step is 
the following matrix transformation: 
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It is assumed that the pivot element, a, is nonzero. For example, a pivot step 
applied to matrix ( 12 ), with the element 10 in row 2 column 1 as pivot, leads to 


/ —5 0 -100 0\ 

0.1 0 2 0 

0 0 0 0 

V 3 0 05/ 









304 INFORMATION STRUCTURES 


2 . 2.6 



Our goal is to design an algorithm that performs this pivot operation on 
sparse matrices that are represented as in Fig. 14. It is clear that the trans- 
formation (13) affects only those rows of a matrix for which there is a nonzero 
element in the pivot column, and it affects only those columns for which there 
is a nonzero entry in the pivot row. 

The pivoting algorithm is in many ways a straightforward application of 
linking techniques we have already discussed; in particular, it bears strong 
resemblances to Algorithm 2.2.4A for addition of polynomials. There are two 
things, however, that make the problem a little tricky: If in (13) we have 6^0 
and c^O but d = 0, the sparse matrix representation has no entry for d and we 
must insert a new entry; and if b ± 0, c / 0, d / 0, but d - be/ a — 0, we must 
delete the entry that was formerly there. These insertion and deletion operations 
are more interesting in a two-dimensional array than in the one-dimensional case; 
to do them we must know what links are affected. Our algorithm processes the 
matrix rows successively from bottom to top. The efficient ability to insert and 
delete involves the introduction of a set of pointer variables PTR[j] , one for each 
column considered; these variables traverse the columns upwards, giving us the 
ability to update the proper links in both dimensions. 

Algorithm S ( Pivot step in a sparse matrix). Given a matrix represented as 
in Fig. 14, we perform the pivot operation (13). Assume that PIVOT is a link 
variable pointing to the pivot element. The algorithm makes use of an auxiliary 
table of link variables PTR[j] , one for each column of the matrix. The variable 
ALPHA and the VAL field of each node are assumed to be floating point or rational 
quantities, while everything else in this algorithm has integer values. 

51. [Initialize.] Set ALPHA 4- 1.0/VAL(PIV0T) , VAL(PIVOT) f- 1.0, and 

10 4- ROW (PIVOT) , P0 4- L0C (BASER0W [10] ) ; 

JO 4— COL (PIVOT) , qo 4- L0C (BASEC0L [JO] ) . 

52. [Process pivot row.] Set PO 4- LEFT (P0) , J 4- C0L(P0). If J < 0, go on 
to step S3 (the pivot row has been traversed). Otherwise set PTR[J] 4- 
L0C (BASEC0L [J] ) and VAL(PO) 4— ALPHA x VAL(PO), and repeat step S2. 

53. [Find new row.] Set Q0 4- UP(Q0). (The remainder of the algorithm deals 
successively with each row, from bottom to top, for which there is an entry 
in the pivot column.) Set I 4- ROW(QO). If I < 0, the algorithm terminates. 
If I = 10, repeat step S3 (we have already done the pivot row). Otherwise 
set P 4— L0C (BASER0W [I] ) , PI 4— LEFT (P) . (The pointers P and PI will now 
proceed across row I from right to left, as P0 goes in synchronization across 
row 10; Algorithm 2.2.4A is analogous. We have P0 = L0C(BASER0W[I0] ) 
at this point.) 

54. [Find new column.] Set P0 4- LEFT(PO) , J 4- C0L(P0). If J < 0, set 

VAL(QO) 4 ALPHA X VAL(QO) and return to S3. If J = JO, repeat step S4. 

(Thus we process the pivot column entry in row I after all other column 
entries have been processed; the reason is that VAL(QO) is needed in step S7.) 


miti. 
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55. [Find I, J element.] If C0L(P1) > J, set P 4- PI, PI 4- LEFT(P), and repeat 
step S5. If COL (PI) = J, go to step S7. Otherwise go to step S6 (we need 
to insert a new element in column J of row I). 

56. [Insert I, J element.] If ROW (UP (PTR [ J] ) ) > I, set PTR[J] 4- UP(PTR[J]). 
and repeat step S6. (Otherwise, we will have R0W(UP(PTR[J] ) ) < I; the new 
element is to be inserted just above NODE (PTR [J] ) in the vertical dimension, 
and just left of NODE(P) in the horizontal dimension.) Otherwise set X -$= 
AVAIL, VAL(X) 4- 0, ROW(X) 4- I, COL(X) 4- J, LEFT(X) 4- PI, UP(X) 4- 
UP (PTR [J] ) , LEFT(P) 4- X, UP(PTR[J]) 4— X, PI 4— X. 

57. [Pivot.] Set VAL(Pl) 4- VAL(Pl) - VAL(QO) x VAL(PO) . IfnowVAL(Pl) = 0, 
go to S8. (Note: When floating point arithmetic is being used, this test 
"VAL (PI) = 0” should be replaced by “|VAL(P1)| < EPSILON” or better yet 
by the condition “most of the significant figures of VAL (PI) were lost in the 
subtraction.”) Otherwise, set PTR[J] 4- PI, P 4- PI, PI 4- LEFT(P) , and go 
back to S4. 

58. [Delete I, J element.] If UP (PTR [J]) / PI (or, what is essentially the same 

thing, if ROW (UP (PTR [J])) > I), set PTR[J] 4- UP(PTR[J]) and repeat 
step S8; otherwise, set UP(PTR[J]) 4- UP(P1), LEFT(P) 4- LEFT(Pl) , 
AVAIL 4= PI, PI 4— LEFT (P) . Go back to S4. | 

The programming of this algorithm is left as a very instructive exercise for 
the reader (see exercise 15). It is worth pointing out here that it is necessary to 
allocate only one word of memory to each of the nodes BASEROW [j] , BASECOL [j] , 
since most of their fields are irrelevant. (See the shaded areas in Fig. 14, and see 
the program of Section 2.2.5.) Furthermore, the value -PTR[j] can be stored as 
ROW (LOC (BASECOL [ )] ) ) for additional storage space economy. The running time 
of Algorithm S is very roughly proportional to the number of matrix elements 
affected by the pivot operation. 

This representation of sparse matrices via orthogonal circular lists is in- 
structive, but numerical analysts have developed better methods. See Fred G. 
Gustavson, ACM Trans, on Math. Software 4 (1978), 250-269; see also the graph 
and network algorithms in Chapter 7. 

EXERCISES 

1. [17] Give a formula for L0C(A[J,K]) if A is the matrix of (l), and if each node 
of the array is two words long, assuming that the nodes are stored consecutively in 
lexicographic order of the indices. 

► 2. [21] Formulas (5) and (6) have been derived from the assumption that 0 < I r < d r 

for 1 < r < k. Give a general formula that applies to the case l T < I r < u r , where 
l r and u r are any lower and upper bounds on the dimensionality. 

3. [21] The text considers lower triangular matrices A \j ,kl for 0 < k < j < n. How 
can the discussion of such matrices readily be modified for the case that subscripts 
start at 1 instead of 0, so that 1 < k < j < n? 

4. [22] Show that if we store the upper triangular array A Ij ,k~} for 0 < j < k < n 
in lexicographic order of the indices, the allocation satisfies the condition of Eq. (8). 
Find a formula for L0C(A[J,K]) in this sense. 
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5. [20] Show that it is possible to bring the value of A[J,K] into register A in one 
MIX instruction, using the indirect addressing feature of exercise 2. 2. 2-3, even when 
A is a triangular matrix as in ( 9 ). (Assume that the values of J and K are in index 
registers.) 

► 6 . [M24] Consider the “tetrahedral arrays” A B [i,j,k], where 0 < k < j < 
i < n in A, and 0 < i < j < k < n in B. Suppose that both of these arrays are 
stored in consecutive memory locations in lexicographic order of the indices; show that 
L0C(A[I,J,K]) = 00 + /i(I) + /2 C J) + / 3 (H) for certain functions / 1 , / 2 , / 3 . Can 
L0C(B [I , J ,K] ) be expressed in a similar manner? 

7 . [M23] Find a general formula to allocate storage for the fc-dimensional tetrahedral 
array A [ii , 12 , . . . ,ikJ , where 0 < ij, < • • • < i 2 < i\ < n. 

8. [33] (P. Wegner.) Suppose we have six tetrahedral arrays A[I,J,K], B[I,J,K], 
C [I , J ,K] , D [I , J ,K] , E [I , J ,K] , and F [I , J ,K] to store in memory, where 0 < K < J < 
I < n. Is there a neat way to accomplish this, analogous to ( 10 ) in the two-dimensional 
case? 

9. [22] Suppose a table, like that indicated in Fig. 13 but much larger, has been 
set up so that all links go in the same direction as shown there (namely, LINK(X) < X 
for all nodes and links). Design an algorithm that finds the addresses of all blue-eyed 
blonde women of ages 21 through 23, by going through the various link fields in such 
a way that upon completion of the algorithm at most one pass has been made through 
each of the lists FEMALE, A21, A22, A23, BLOND, and BLUE. 

10. [26] Can you think of a better way to organize a personnel table so that searches 
as described in the previous exercise would be more efficient? (The answer to this 
exercise is not merely “yes” or “no.”) 

11. [11] Suppose that we have a 200 X 200 matrix in which there are at most four 
nonzero entries per row. How much storage is required to represent this matrix as in 
Fig. 14, if we use three words per node except for list heads, which will use one word? 

► 12. [20] What are VAL(QO), VAL(PO), and VAL(Pl) at the beginning of step S7, in 
terms of the notation a, 6 , c, d used in ( 13 )? 

► 13. [22] Why were circular lists used in Fig. 14 instead of straight linear lists? Could 
Algorithm S be rewritten so that it does not make use of the circular linkage? 

14. [22] Algorithm S actually saves pivoting time in a sparse matrix, since it avoids 
consideration of those columns in which the pivot row has a zero entry. Show that 
this savings in running time can be achieved in a large sparse matrix that is stored 
sequentially, with the help of an auxiliary table LINK[j] , 1 < j < n. 

► 15. [29] Write a MIXAL program for Algorithm S. Assume that the VAL field is a 
floating point number, and that Mix’s floating point arithmetic operators FADD. FSUB. 
FMUL, and FDIV can be used for operations on this field. Assume for simplicity that 
FADD and FSUB return the answer zero when the operands added or subtracted cancel 
most of the significance, so that the test “VAL (PI) = 0” may safely be used in step S7. 
The floating point operations use only rA, not rX. 

16. [25] Design an algorithm to copy a sparse matrix. (In other words, the algorithm 
is to yield two distinct representations of a matrix in memory, having the form of 
Fig. 14, given just one such representation initially.) 

17. [26] Design an algorithm to multiply two sparse matrices; given matrices A and B. 
form a new matrix C, where CLi,jl = Ylk AK,fc]B[fc,j] . The two input matrices and 
the output matrix should be represented as in Fig. 14. 
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18. [22] The following algorithm replaces a matrix by the inverse of that matrix, 
assuming that the entries are A Li.jl, for 1 < i, j < n: 

i) For k = 1,2, ... ,n do the following: Search row k in all columns not yet used 
as a pivot column, to find an entry with the greatest absolute value; set C [fc] equal to 
the column in which this entry was found, and do a pivot step with this entry as pivot. 
(If all such entries are zero, the matrix is singular and has no inverse.) 

ii) Permute rows and columns so that what was row k becomes row C [A:] , and 
what was column C[fc] becomes column k. 

The problem in this exercise is to use the stated algorithm to invert the matrix 

12 3 

0 1 2 
0 0 1 

by hand calculation. 

19. [31] Modify the algorithm described in exercise 18 so that it obtains the inverse 
of a sparse matrix that is represented in the form of Fig. 14. Pay special attention to 
making the row- and column-permutation operations of step (ii) efficient. 

20. [20] A tridiagonal matrix has entries a,j that are zero except when \i - j\ < 1, 
for 1 < i, j < n. Show that there is an allocation function of the form 

L0C(A[I , J] ) — do + ctil -}- CI 2 J, |l — J| < 1, 

which represents all of the relevant elements of a tridiagonal matrix in (3n — 2) consec- 
utive locations. 

21. [20] Suggest a storage allocation function for n x n matrices where n is variable. 
The elements A [I , J] for 1 < I, J < n should occupy n 2 consecutive locations, regardless 
of the value of n. 

22. [M25] (P. Chowla, 1961.) Find a polynomial p(ii, . . . ,ik) that assumes each 
nonnegative integer value exactly once as the indices (ii,...,i k ) run through all k- 

dimensional nonnegative integer vectors, with the additional property that i x -\ f- i k < 

ji H Vjk implies p{i u ...,i k ) < p(j u , . . , j k ). 

23. [23] An extendible matrix is initially lxl, then it grows from size to x n either 
to size (to + 1) x n or to size to x (n + 1) by adding either a new row or a new column. 
Show that such a matrix can be given a simple allocation function in which the elements 
A[I,J] occupy mn consecutive locations, for 0 < I < to and 0 < J < n; no elements 
change location when the matrix grows. 

► 24. [25] ( The sparse array trick.) Suppose you want to use a large array for random 
access, although you won’t actually be referring to very many of its entries. You want 
A[fc] to be zero the first time you access it, yet you don’t want to spend the time to 
set every location to zero. Explain how it is possible to read and write any desired 
elements A[fc] reliably, given k, without assuming anything about the actual initial 
memory contents, by doing only a small fixed number of additional operations per 
array access. 
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2.3. TREES 

We NOW TURN to a study of trees, the most important nonlinear structures 
that arise in computer algorithms. Generally speaking, tree structure means a 
“branching” relationship between nodes, much like that found in the trees of 
nature. 

Let us define a tree formally as a finite set T of one or more nodes such that 

a) there is one specially designated node called the root of the tree, root(T); 
and 

b) the remaining nodes (excluding the root) are partitioned into m > 0 disjoint 
sets Ti, . . T m , and each of these sets in turn is a tree. The trees T) , . . . , T m 
are called the subtrees of the root. 

The definition just given is recursive: We have defined a tree in terms of 
trees. Of course, there is no problem of circularity involved here, since trees 
with one node must consist of only the root, and trees with n > 1 nodes are 
defined in terms of trees with fewer than n nodes; hence the concept of a tree 
with two nodes, three nodes, or ultimately any number of nodes, is determined 
by the definition given. There are nonrecursive ways to define trees (for example, 
see exercises 10. 12, and 14, and Section 2.3.4), but a recursive definition seems 
most appropriate since recursion is an innate characteristic of tree structures. 
The recursive character of trees is present also in nature, since buds on young 
trees eventually grow into subtrees with buds of their own, and so on. Exercise 3 
illustrates how to give rigorous proofs of important facts about trees based on a 
recursive definition such as the one above, by using induction on the number of 
nodes in a tree. 

It follows from our definition that every node of a tree is the root of some 
subtree contained in the whole tree. The number of subtrees of a node is called 
the degree of that node. A node of degree zero is called a terminal node, or 
sometimes a leaf. A nonterminal node is often called a branch node. The level 
of a node with respect to T is defined recursively: The level of root(T) is zero, 
and the level of any other node is one higher than that node’s level with respect 
to the subtree of root(T) containing it. 

These concepts are illustrated in Fig. 15, which shows a tree with seven 
nodes. The root is A, and it has the two subtrees {£?} and { C , D, E , F, G}. The 
tree {C, D , E , F, G} has node G as its root. Node G is on level 1 with respect to 
the whole tree, and it has three subtrees {£>}, {£}, and {F, G}; therefore G has 
degree 3. The terminal nodes in Fig. 15 are B, D, E, and G; F is the only node 
with degree 1; G is the only node with level 3. 

If the relative order of the subtrees Ti,...,T m in (b) of the definition is 
important, we say that the tree is an ordered tree ; when m > 2 in an ordered 
tree, it makes sense to call T 2 the “second subtree” of the root, etc. Ordered trees 
are also called “plane trees” by some authors, since the manner of embedding 
the tree in a plane is relevant. If we do not care to regard two trees as different 
when they differ only in the respective ordering of subtrees of nodes, the tree 
is said to be oriented, since only the relative orientation of the nodes, not their 
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Fig. 15. A tree. Fig. 16. Another tree. 


order, is being considered. The very nature of computer representation defines 
an implicit ordering for any tree, so in most cases ordered trees are of greatest 
interest to us. We will therefore tacitly assume that all trees we discuss are 
ordered, unless explicitly stated otherwise. Accordingly, the trees of Figs. 15 
and 16 will generally be considered to be different, although they would be the 
same as oriented trees. 

A forest is a set (usually an ordered set) of zero or more disjoint trees. 
Another way to phrase part (b) of the definition of tree would be to say that the 
nodes of a tree excluding the root form a forest. 

There is very little distinction between abstract forests and trees. If we 
delete the root of a tree, we have a forest; conversely, if we add just one node to 
any forest and regard the trees of the forest as subtrees of the new node, we get a 
tree. Therefore the words tree and forest are often used almost interchangeably 
during informal discussions about data structures. 



Trees can be drawn in many ways. Besides the diagram of Fig. 15, three of 
the principal alternatives are shown in Fig. 17, depending on where the root is 
placed. It is not a frivolous joke to worry about how tree structures are drawn 
in diagrams, since there are many occasions in which we want to say that one 
node is “above” or “higher than” another node, or to refer to the “rightmost” 
element, etc. Certain algorithms for dealing with tree structures have become 
known as “top down” methods, as opposed to “bottom up.” Such terminology 
leads to confusion unless we adhere to a uniform convention for drawing trees. 

It may seem that the form of Fig. 15 would be preferable simply because that 
is how trees grow in nature; in the absence of any compelling reason to adopt 
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any of the other three forms, we might as well adopt nature’s time-honored 
tradition. With this in mind, the author consistently followed a root-at-the- 
bottom convention as the present set of books was first being prepared, but after 
two years of trial it was found to be a mistake: Observations of the computer 
literature and numerous informal discussions with computer scientists about a 
wide variety of algorithms showed that trees were drawn with the root at the 
top in more than 80 percent of the cases examined. There is an overwhelming 
tendency to make hand-drawn charts grow downwards instead of upwards (and 
this is easy to understand in view of the way we write); even the word “subtree,” 
as opposed to “supertree,” tends to connote a downward relationship. From 
these considerations we conclude that Fig. 15 is upside down. Henceforth we 
will almost always draw trees as in Fig. 17(b), with the root at the top and 
leaves at the bottom. Corresponding to this orientation, we should perhaps call 
the root node the apex of the tree, and speak of nodes at shallow and deep levels. 

It is necessary to have good descriptive terminology for talking about trees. 
Instead of making somewhat ambiguous references to “above” and “below,” we 
generally use genealogical words taken from the terminology of family trees. 
Figure 18 shows two common types of family trees. The two types are quite 
different: A pedigree shows the ancestors of a given individual, while a lineal 
chart shows the descendants. 

If “cross-breeding” occurs, a pedigree is not really a tree, because different 
branches of a tree (as we have defined it) can never be joined together. To 
compensate for this discrepancy, Fig. 18(a) mentions Queen Victoria and Prince 
Albert twice in the sixth generation; King Christian IX and Queen Louise 
actually appear in both the fifth and sixth generations. A pedigree can be 
regarded as a true tree if each of its nodes represents “a person in the role of 
mother or father of so-and-so,” not simply a person as an individual. 

Standard terminology for tree structures is taken from the second form of 
family tree, the lineal chart: Each root is said to be the parent of the roots of 
its subtrees, and the latter are said to be siblings ; they are children of their 
parent. The root of the entire tree has no parent. For example, in Fig. 19, 
C has three children, D , E, and F; E is the parent of G; B and C are siblings. 
Extension of this terminology — for example, A is the great-grandparent of G ; 
B is an aunt or uncle of F; H and F are first 
cousins — is clearly possible. Some authors use the 
masculine designations “father, son, brother” in- 
stead of “parent, child, sibling” ; others use “mother, 
daughter, sister.” In any case a node has at most 
one parent or progenitor. We use the words ancestor 
and descendant to denote a relationship that may 
span several levels of the tree: The descendants of C 
in Fig. 19 are D, E, F, and G; the ancestors of G 
are E, C, and A. 

The pedigree in Figure 18(a) is an example of a binary tree, which is another 
important type of tree structure. The reader has undoubtedly seen binary trees 



Fig. 19. Conventional 
tree diagram. 
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in connection with tennis tournaments or other sporting events. In a binary tree 
each node has at most two subtrees; and when only one subtree is present, we 
distinguish between the left and right subtree. More formally, let us define a 
binary tree as a finite set of nodes that either is empty, or consists of a root 
and the elements of two disjoint binary trees called the left and right subtrees 
of the root. 

This recursive definition of binary tree should be studied carefully. Notice 
that a binary tree is not a special case of a tree; it is another concept entirely 
(although we will see many relations between the two concepts). For example, 
the binary trees 


U) 


U) 


and 


are distinct — the root has an empty right subtree in one case and a nonempty 
right subtree in the other — although as trees these diagrams would represent 
identical structures. A binary tree can be empty; a tree cannot. Therefore we 
will always be careful to use the word “binary” to distinguish between binary 
trees and ordinary trees. Some authors define binary trees in a slightly different 
manner (see exercise 20). 

Tree structure can be represented graphically in several other ways bearing 
no resemblance to actual trees. Figure 20 shows three diagrams that reflect the 
structure of Fig. 19: Figure 20(a) essentially represents Fig. 19 as an oriented 
tree; this diagram is a special case of the general idea of nested sets , namely 
a collection of sets in which any pair of sets is either disjoint or one contains 
the other. (See exercise 10.) Part (b) of the figure shows nested sets in a line, 
much as part (a) shows them in a plane; in part (b) the ordering of the tree is 
also indicated. Part (b) may also be regarded as an outline of an algebraic 
formula involving nested parentheses. Part (c) shows still another common 
way to represent tree structure, using indentation. The number of different 
representation methods in itself is ample evidence for the importance of tree 
structures in everyday life as well as in computer programming. Any hierarchical 
classification scheme leads to a tree structure. 



Fig. 20. Further ways to show tree structure: (a) nested sets; (b) nested parentheses; 
(c) indentation. 
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Aii algebraic formula defines an implicit tree structure that is often conveyed 
by other means instead of, or in addition to, the use of parentheses. For example, 
Figure 21 shows a tree corresponding to the arithmetic expression 

a - b{c/d - c/f). ( 2 ) 

Standard mathematical conventions, according to which multiplication and divi- 
sion take precedence over addition and subtraction, allow us to use a simplified 
form like ( 2 ) instead of the fully parenthesized form “a — (b x ((c/d) + (e//)))”. 
This connection between formulas and trees is very important in applications. 



Fig. 21. Tree representation of formula ( 2 ). 

Notice that the indented list in Fig. 20(c) looks very much like the table 
of contents in a book. Indeed, this book itself has a tree structure; the tree 
structure of Chapter 2 is shown in Fig. 22. Here we notice a significant idea: 
The method used to number sections in this book is another way to specify tree 
structure. Such a method is often called “Dewey decimal notation” for trees, by 
analogy with the similar classification scheme of this name used in libraries. The 
Dewey decimal notation for the tree of Fig. 19 is 

1 A; 1.1 B- 1.1.1 H; 1.1.2 J; 1.2 C; 

1.2.1 D; 1.2.2 E; 1.2. 2.1 G; 1.2.3 F. 

Dewey decimal notation applies to any forest: The root of the fcth tree in the 
forest is given number k; and if a is the number of any node of degree m, its 
children are numbered a.l, a. 2, . . . , a.m. The Dewey decimal notation satisfies 
many simple mathematical properties, and it is a useful tool in the analysis 
of trees. One example of this is the natural sequential ordering it gives to the 
nodes of an arbitrary tree, analogous to the ordering of sections within this book. 
Section 2.3 precedes Section 2.3.1, and follows Section 2.2.6. 

There is an intimate relation between Dewey decimal notation and the 
notation for indexed variables that we have already been using extensively. If 
F is a forest of trees, we may let F[l] denote the subtrees of the first tree, so 
that F[l][2] = .F[l,2] stands for the subtrees of the second subtree of F[ 1], and 
F[ 1.2, 1] stands for the first subforest of the latter, and so on. Node a.b.c.d in 
Dewey decimal notation is the parent of F[a, b , c, d\. This notation is an extension 
of ordinary index notation, because the admissible range of each index depends 
on the values in the preceding index positions. 
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Thus, in particular, we see that any rectangular array can be thought of 
as a special case of a tree or forest structure. For example, here are two 
representations of a 3 x 4 matrix: 


(A[ 1 , 1 ] A[l,2] 41,3] A[l, 4] \ 
42,1] A[2, 2] A[2, 3] A[2,A}\ 
\A[3,1] A[3,2] A[3,3] 43,4]/ 




CM . CO 

/IW _//\\ 



^ ^ ^ 


t— 1 CM CO ^ 
CM CM CM" CM 
^ ^ ^ ^ 


rH (M ro 
co co co co 
^ ^ ^ ^ 


It is important to observe, however, that this tree structure does not faithfully 
reflect all of the matrix structure; the row relationships appear explicitly in the 
tree but the column relationships do not. 

A forest can, in turn, be regarded as a special case of what is commonly 
called a list structure. The word “list” is being used here in a very technical 
sense, and to distinguish the technical use of the word we will always capitalize it: 
"List.” A List is defined (recursively) as a finite sequence of zero or more atoms 
or Lists. Here “atom” is an undefined concept referring to elements from any 
universe of objects that might be desired, so long as it is possible to distinguish 
an atom from a List. By means of an obvious notational convention involving 
commas and parentheses, we can distinguish between atoms and Lists and we 
can conveniently display the ordering within a List. As an example, consider 


L = (a, (b,a,b), ( ), c, ((( 2 )))), ( 3 ) 

which is a List with five elements: first the atom a, then the List (b,a,b), then 
the empty List ( ), then the atom c, and finally the List (((2))). The latter List 
consists of the List ((2)), which consists of the List (2), which consists of the 
atom 2 . 

The following tree structure corresponds to L: 



* 

I 


2 

The asterisks in this diagram indicate the definition and appearance of a List, 
as opposed to the appearance of an atom. Index notation applies to Lists as it 
does to forests; for example, L[ 2] = ( 6 , a, b), and L[2,2] = a. 

No data is carried in the nodes for the Lists in ( 4 ) other than the fact that 
they are Lists. But it is possible to label the nonatomic elements of Lists with 
information, as we have done for trees and other structures; thus 


A = (a: (b, c), d:( )) 
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would correspond to a tree that we can draw as follows: 


d* 


The big difference between Lists and trees is that Lists may overlap (that 
is, sub-Lists need not be disjoint) and they may even be recursive (may contain 
themselves). The List 

M = ( M ) ( 5 ) 

corresponds to no tree structure, nor does the List 


N = (a:M, b:M, c, N). 


(6) 


(In these examples, capital letters refer to Lists, lowercase letters to labels and 
atoms.) We might diagram (5) and (6) as follows, using an asterisk to denote 
each place where a List is defined: 


*[N] 

a*[M] b[M } c 

[M] 


[IV] 


(7) 


Actually, Lists are not so complicated as the examples above might indicate. 
They are, in essence, a rather simple generalization of the linear lists that we 
have considered in Section 2.2, with the additional proviso that the elements of 
linear Lists may be link variables that point to other linear Lists (and possibly 
to themselves). 

Summary: Four closely related kinds of information structures — trees, for- 
ests, binary trees, and Lists — arise from many sources, and they are therefore 
important in computer algorithms. We have seen various ways to diagram these 
structures, and we have considered some terminology and notations that are 
useful in talking about them. The following sections develop these ideas in 
greater detail. 


EXERCISES 

1. [18] How many different trees are there with three nodes, A, B, and C? 

2. [20] How many different oriented trees are there with three nodes, A, B. and C? 

3. [M20] Prove rigorously from the definitions that for every node X in a tree there is 
a unique path up to the root, namely a unique sequence of k > 1 nodes Xi,X 2 ,..., X k 
such that Xj is the root of the tree, X k = X, and Xj is the parent of Xj+ 1 for 1 < j < k. 
(This proof will be typical of the proofs of nearly all the elementary facts about tree 
structures.) Hint: Use induction on the number of nodes in the tree. 

4. [01] True or false: In a conventional tree diagram (root at the top), if node X has 
a higher level number than node Y, then node X appears lower in the diagram than 
node Y. 
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5. [02] If node A has three siblings and B is the parent of A, what is the degree 
of B? 

► 6. [21] Define the statement “ X is an mth cousin of Y, n times removed” as a 
meaningful relation between nodes X and Y of a tree, by analogy with family trees, if 
m > 0 and n > 0. (See a dictionary for the meaning of these terms in regard to family 
trees.) 

7. [23] Extend the definition given in the previous exercise to all m > — 1 and to all 
integers n > — (m + 1) in such a way that for any two nodes X and Y of a tree there 
are unique m and n such that X is an mth cousin of Y, n times removed. 

► 8. [03] What binary tree is not a tree? 

9. [00] In the two binary trees of (i), which node is the root ( B or A )? 

10. [M20] A collection of nonempty sets is said to be nested if, given any pair X, Y 
of the sets, either X C Y or X 3 7 or A' and Y are disjoint. (In other words, X IT Y 
is either X, Y. or 0.) Figure 20(a) indicates that any tree corresponds to a collection 
of nested sets; conversely, does every such collection correspond to a tree? 

► 11. [HM32] Extend the definition of tree to infinite trees by considering collections of 
nested sets as in exercise 10. Can the concepts of level, degree, parent, and child be 
defined for each node of an infinite tree? Give examples of nested sets of real numbers 
that correspond to a tree in which 

a) every node has uncountable degree and there are infinitely many levels; 

b) there are nodes with uncountable level; 

c) every node has degree at least 2 and there are uncount.ably many levels. 

12. [M23] Under what conditions does a partially ordered set correspond to an un- 
ordered tree or forest? (Partially ordered sets are defined in Section 2.2.3.) 

13. [10] Suppose that node X is numbered ai.d2. • • • .at, in the Dewey decimal system; 
what are the Dewey numbers of the nodes in the path from X to the root (see 
exercise 3)? 

14. [M22] Let S be any nonempty set of elements having the form “l.ai. ■■■.a*,”, 
where k > 0 and a.i , . . . , a k are positive integers. Show that S specifies a tree when it 
is finite and satisfies the following condition: “If a.m is in the set, then so is a.(m — 1) 
if m > 1, or a if m = 1.” (This condition is clearly satisfied in the Dewey decimal 
notation for a tree; therefore it is another way to characterize tree structure.) 

► 15. [20] Invent a notation for the nodes of binary trees, analogous to the Dewey 
decimal notation for nodes of trees. 

16. [20] Draw trees analogous to Fig. 21 corresponding to the arithmetic expressions 
(a) 2(a — b/c ); (b) a + b + 5c. 

17. [01] If Z stands for Fig. 19 regarded as a forest, what node is parent(Z[l, 2, 2])? 

18. [08] In List (3), what is L[ 5, 1, 1]? What is L[ 3, 1]? 

19. [15] Draw a List diagram analogous to (7) for the List L = (a, (L)). What, is L[ 2] 
in this List? What is L[ 2, 1, 1]? 

► 20. [M21] Define a 0 -2-tree as a tree in which each node has exactly zero or two 
children. (Formally, a 0-2-tree consists of a single node, called its root, plus 0 or 2 
disjoint 0-2-trees.) Show that every 0-2-tree has an odd number of nodes; and give a 
one-to-one correspondence between binary trees with n nodes and (ordered) 0-2-trees 
with 2n + 1 nodes. 
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21 . [M22] If a tree has m nodes of degree 1, n 2 nodes of degree 2, . . . , and n m nodes 
of degree m, how many terminal nodes does it have? 

► 22 . [21] Standard European paper sizes AO, Al, A2, . . . , An, . . . are rectangles whose 
sides are in the ratio \/2 to 1 and whose areas are 2~" square meters. Therefore if we 
cut a sheet of An paper in half, we get two sheets of A(n+ 1) paper. Use this principle 
to design a graphic representation of binary trees, and illustrate your idea by drawing 
the representation of 2.3.1-(i) below. 


2.3.1. Traversing Binary Trees 

It is important to acquire a good understanding of the properties of binary 
trees before making further investigations of trees, since general trees are usually 
represented in terms of some equivalent binary tree inside a computer. 

We have defined a binary tree as a finite set of nodes that either is empty, 
or consists of a root together with two binary trees. This definition suggests a 
natural way to represent binary trees within a computer: We can have two links, 
LLINK and RLINK, within each node, and a link variable T that is a “pointer to 
the tree.” If the tree is empty, T = A; otherwise T is the address of the root node 
of the tree, and LLINK(T) , RLINK(T) are pointers to the left and right subtrees of 
the root, respectively. These rules recursively define the memory representation 
of any binary tree; for example, 



is represented by 



This simple and natural memory representation accounts for the special 
importance of binary tree structures. We will see in Section 2.3.2 that general 
trees can conveniently be represented as binary trees. Moreover, many trees 
that arise in applications are themselves inherently binary, so binary trees are of 
interest in their own right. 
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There are many algorithms for manipulation of tree structures, and one 
idea that occurs repeatedly in these algorithms is the notion of traversing or 
“walking through” a tree. This is a method of examining the nodes of the tree 
systematically so that each node is visited exactly once. A complete traversal 
of the tree gives us a linear arrangement of the nodes, and many algorithms are 
facilitated if we can talk about the “next” node following or preceding a given 
node in such a sequence. 

Three principal ways may be used to traverse a binary tree: We can visit 
the nodes in preorder , inorder , or postorder. These three methods are defined 
recursively. When the binary tree is empty, it is “traversed” by doing nothing; 
otherwise the traversal proceeds in three steps: 

Preorder traversal Inorder traversal 

Visit the root Traverse the left subtree 

Traverse the left subtree Visit the root 

Traverse the right subtree Traverse the right subtree 

Postorder traversal 
Traverse the left subtree 
Traverse the right subtree 
Visit the root 

If we apply these definitions to the binary tree of (1) and (2), we find that the 
nodes in preorder are 

ABDCEGFHJ. (3) 

(First comes the root A, then comes the left subtree 



in preorder, and finally we traverse the right subtree in preorder.) For inorder we 
visit the root between visits to the nodes of each subtree, essentially as though 
the nodes were “projected” down onto a single horizontal line, and this gives the 
sequence 

DBAEGCHFJ. (4) 

The postorder for the nodes of this binary tree is, similarly, 

DBGEHJFC A. (5) 

We will see that these three ways of arranging the nodes of a binary tree into 
a sequence are extremely important, as they are intimately connected with most 
of the computer methods for dealing with trees. The names preorder , inorder , 
and postorder come, of course, from the relative position of the root with respect 
to its subtrees. In many applications of binary trees, there is symmetry between 
the meanings of left subtrees and right subtrees, and in such cases the term 
symmetric order is used as a synonym for inorder. Inorder, which puts the root 
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P<— LLINK(P) 



Fig. 23. Algorithm T for inorder traversal. 

m the middle, is essentially symmetric between left and right: If the binary tree 
is reflected about a vertical axis, the symmetric order is simply reversed. 

A recursively stated definition, such as the one just given for the three basic 
orders, must be reworked in order to make it directly applicable to computer 
implementation. General methods for doing this are discussed in Chapter 8; we 
usually make use of an auxiliary stack, as in the following algorithm: 

Algorithm T ( Traverse binary tree in inorder). Let T be a pointer to a binary 
tree having a representation as in (2); this algorithm visits all the nodes of the 
binary tree in inorder, making use of an auxiliary stack A. 

Tl. [Initialize.] Set stack A empty, and set the link variable P i — T. 

T2. [P = A?] If P = A, go to step T4. 

T3. [Stack -t= P.] (Now P points to a nonempty binary tree that is to be 
traversed.) Set A <S= P: that is, push the value of P onto stack A. (See 
Section 2.2.1.) Then set P <- LLINK(P) and return to step T2. 

T4. [P •<= Stack.] If stack A is empty, the algorithm terminates; otherwise set 

P <J= A. 

T5. [Visit P.] Visit NODE(P). Then set P RLINK(P) and return to step T2. | 

In the final step of this algorithm, the word “visit” means that we do 
whatever activity is intended as the tree is being traversed. Algorithm T runs like 
a coroutine with respect to this other activity: The main program activates the 
coroutine whenever it wants P to move from one node to its inorder successor. 
Of course, since this coroutine calls the main routine in only one place, it is 
not much different from a subroutine (see Section 1.4.2). Algorithm T assumes 
that the external activity deletes neither NODE(P) nor any of its ancestors from 
the tree. 

The reader should now attempt to play through Algorithm T using the 
binary tree ( 2 ) as a test case, in order to see the reasons behind the procedure. 
When we get to step T3, we want to traverse the binary tree whose root is 
indicated by pointer P. The idea is to save P on a stack and then to traverse the 
left subtree; when this has been done, we will get to step T4 and will find the 
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old value of P on the stack again. After visiting the root, NODE(P), in step T5, 
the remaining job is to traverse the right subtree. 

Algorithm T is typical of many other algorithms that we will see later, so 
it is instructive to look at a formal proof of the remarks made in the preceding 
paragraph. Let us now attempt to prove that Algorithm T traverses a binary tree 
of n nodes in inorder, by using induction on n. Our goal is readily established if 
we can prove a slightly more general result: 

Starting at step T2 with P a pointer to a binary tree of n nodes and with 
the stack A containing A [1] ... A [m] for some m > 0, the procedure of steps 
T2-T5 will traverse the binary tree in question, in inorder, and will then 
arrive at step T4 with stack A returned to its original value A [1] . . . A[m] . 

This statement is obviously true when n = 0, because of step T2. If n > 0, 
let Po be the value of P upon entry to step T2. Since Po y! A, we will perform 
step T3, which means that stack A is changed to A [1] . . . A[?n]P 0 and P is set 
to LLINK(Po). Now the left subtree has fewer than n nodes, so by induction we 
will traverse the left subtree in inorder and will ultimately arrive at step T4 with 
A [1] . . . A[m] Po on the stack. Step T4 returns the stack to A [1] . . .A [to] and 
sets P «— P () . Step T5 now visits NODE(P 0 ) and sets P «— RLINK(P 0 ). Now the 
right subtree has fewer than n nodes, so by induction we will traverse the right 
subtree in inorder and arrive at step T4 as required. The tree has been traversed 
in inorder, by the definition of that order. This completes the proof. 

An almost identical algorithm may be formulated that traverses binary trees 
in preorder (see exercise 12). It is slightly more difficult to achieve the traversal 
in postorder (see exercise 13), and for this reason postorder is not as important 
for binary trees as the others are. 

It is convenient to define a new notation for the successors and predecessors 
of nodes in these various orders. If P points to a node of a binary tree, let 

P* = address of successor of NODE(P) in preorder; 

PS = address of successor of NODE(P) in inorder; 

Pf = address of successor of NODE(P) in postorder; 

*P = address of predecessor of NODE(P) in preorder; 

$P = address of predecessor of NODE(P) in inorder; 

pP = address of predecessor of NODE(P) in postorder. 

If there is no such successor or predecessor of NODE(P), the value LOC(T) is 
generally used, where T is an external pointer to the tree in question. We have 
*(P*) = (*P)* = P, $(P$) = ($P)$ = P. and tt(Ptt) = (BP)tt = P. As an example of 
this notation, let INFO(P) be the letter shown in NQDE(P) in the tree ( 2 ); then 
if P points to the root, we have INFO(P) = A, INF0(P*) = B, INFO(Ps) = E, 
INFO (*P) = B , INFO (PP) = C , and PP = *P = LOC(T). 

At this point the reader will perhaps experience a feeling of insecurity about 
the intuitive meanings of P*, P$, etc. As we proceed further, the ideas will 
gradually become clearer; exercise 16 at the end of this section may also be of 
help. The in “P$” is meant to suggest the letter S, for “symmetric order.” 
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There is an important alternative to the memory representation of binary 
trees given in ( 2 ), which is somewhat analogous to the difference between circular 
lists and straight one-way lists. Notice that there are more null links than other 
pointers in the tree ( 2 ), and indeed this is true of any binary tree represented by 
the conventional method (see exercise 14). But we don’t really need to waste all 
that memory space. For example, we could store two “tag” indicators with each 
node, which would tell in just two bits of memory whether or not the LLINK or 
RLINK, or both, are null; the memory space for terminal links could then be used 
for other purposes. 

An ingenious use of this extra space has been suggested by A. J. Perlis 
and C. Thornton, who devised the so-called threaded tree representation. In this 
method, terminal links are replaced by “threads” to other parts of the tree, as 
an aid to traversal. The threaded tree equivalent to ( 2 ) is 


(7) 


Here dotted lines represent the “threads,” which always go to a higher node of the 
tree. Every node now has two links: Some nodes, like C, have two ordinary links 
to left and right subtrees; other nodes, like H, have two thread links; and some 
nodes have one link of each type. The special threads emanating from D and J 
will be explained later. They appear in the “leftmost” and “rightmost” nodes. 

In the memory representation of a threaded binary tree it is necessary to 
distinguish between the dotted and solid links; this can be done as suggested 
above by two additional one-bit fields in each node, LTAG and RTAG. The threaded 
representation may be defined precisely as follows: 



Unthreaded representation Threaded representation 


LLINK (P) = A 
LLINK (P) = Q / A 
RLINK (P) = A 
RLINK (P) = Q / A 


LTAG(P) = 1, 
LTAG(P) = 0, 
RTAG(P) = 1, 
RTAG(P) = 0, 


LLINK (P) = $P 
LLINK(P) = Q 
RLINK(P) = P$ 
RLINK (P) = q 


According to this definition, each new thread link points directly to the 
predecessor or successor of the node in question, in symmetric order (inorder). 
Figure 24 illustrates the general orientation of thread links in any binary tree. 

In some algorithms it can be guaranteed that the root of any subtree always 
will appear in a lower memory location than the other nodes of the subtree. 
Then LTAG(P) will be 1 if and only if LLINK (P) < P, so LTAG will be redundant. 
The RTAG bit will be redundant for the same reason. 
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Fig. 24. General orientation of left and right thread links in a threaded binary tree. 
Wavy lines indicate links or threads to other parts of the tree. 


The great advantage of threaded trees is that traversal algorithms become 
simpler. For example, the following algorithm calculates P$, given P: 

Algorithm S ( Symmetric (inorder) successor in a threaded binary tree). If P 
points to a node of a threaded binary tree, this algorithm sets Q 4— P*. 

51. [RLINK(P) a thread?] Set Q 4- RLINK(P) . If RTAG(P) = 1, terminate the 
algorithm. 

52. [Search to left.] If LTAG(Q) = 0, set Q 4— LLINK(Q) and repeat this step. 
Otherwise the algorithm terminates. | 

Notice that no stack is needed here to accomplish what was done using a 
stack in Algorithm T. In fact, the ordinary representation ( 2 ) makes it impossible 
to find P$ efficiently, given only the address of a random point P in the tree. Since 
no links point upward in an unthreaded representation, there is no clue to what 
nodes are above a given node, unless we retain a history of how we reached that 
point. The stack in Algorithm T provides the necessary history when threads 
are absent. 

We claim that Algorithm S is “efficient,” although this property is not 
immediately obvious, since step S2 can be executed any number of times. In 
view of the loop in step S2, would it perhaps be faster to use a stack after all, 
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as Algorithm T does? To investigate this question, we will consider the average 
number of times that step S2 must be performed if P is a “random” point in 
the tree; or what is the same, we will determine the total number of times that 
step S2 is performed if Algorithm S is used repeatedly to traverse an entire tree. 

At the same time as this analysis is being carried out, it will be instructive 
to study complete programs for both Algorithms S and T. As usual, we should 
be careful to set all of our algorithms up so that they work properly with empty 
binary trees; and if T is the pointer to the tree, we would like to have LOC(T)* 
and L0C(T)$ be the first nodes in preorder or symmetric order, respectively. For 
threaded trees, it turns out that things will work nicely if N0DE(L0C(T) ) is made 
into a “list head” for the tree, with 

LLINK(HEAD) = T, LTAG(HEAD) = 0, 

RLINK (HEAD) = HEAD. RTAG(HEAD) = 0. ^ 

(Here HEAD denotes LOC(T), the address of the list head.) An empty threaded 
tree will satisfy the conditions 

LL INK (HEAD) = HEAD, LTAG(HEAD) = 1. ( 9 ) 

The tree grows by having nodes inserted to the left of the list head. (These 
initial conditions are primarily dictated by the algorithm to compute P*. which 
appears m exercise 17.) In accordance with these conventions, the computer 
representation for the binary tree (l), as a threaded tree, is 





With these preliminaries out of the way, we are now ready to consider MIX 
versions of Algorithms S and T. The following programs assume that binary tree 
nodes have the two-word form 


LTAG 

LLINK 

INF01 

RTAG 

RLINK 

INF02 


In an unthreaded tree, LTAG and RTAG will always be “+” and terminal links will 
be represented by zero. In a threaded tree, we will use “+” for tags that are 0 
and for tags that are 1. The abbreviations LLINKT and RLINKT will be used 
to stand for the combined LTAG-LLINK and RTAG-RLINK fields, respectively. 


V 
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/Ss The two tag bits occupy otherwise-unused sign positions of a MIX word, so 
jl they cost nothing in memory space. Similarly, with the MMIX computer we 
will be able to use the least significant bits of link fields as tag bits that come 
"for free,” because pointer values will generally be even, and because MMIX will 
make it easy to ignore the low-order bits when addressing memory. 

The following two programs traverse a binary tree in symmetric order (that 
is, inorder), jumping to location VISIT periodically with index register 5 pointing 
to the node that is currently of interest. 

Program T. In this implementation of Algorithm T, the stack is kept in loca- 
tions A + 1, A + 2, . . . , A + MAX; rI6 is the stack pointer and rI5 = P. OVERFLOW 
occurs if the stack grows too large. The program has been rearranged slightly 
from Algorithm T (step T2 appears thrice), so that the test for an empty stack 
need not be made when going directly from T3 to T2 to T4. 


01 

LLINK 

EQU 

1:2 



02 

RLINK 

EQU 

1:2 



03 

T1 

LD5 

HEAD (LLINK) 

1 

T 1. Initialize. Set P 4— T. 

04 

T2A 

J5Z 

DONE 

1 

Stop if P = A. 

05 


ENT6 

0 

1 


06 

T3 

DEC6 

MAX 

n 

T3. Stack ■<= P. 

07 


J6NN 

OVERFLOW 

n 

Has that stack reached capacity? 

08 


INC6 

MAX+1 

n 

If not, increase the stack pointer. 

09 


ST5 

A, 6 

n 

Store P in the stack. 

10 


LD5 

0,5 (LLINK) 

n 

P 4- LLINK (P). 

11 

T2B 

J5NZ 

T3 

n 

To T3 if P / A. 

12 

T4 

LD5 

A, 6 

n 

T4. P <= Stack. 

13 


DEC6 

1 

n 

Decrease the stack pointer. 

U 

T5 

JMP 

VISIT 

n 

T5. Visit P. 

15 


LD5 

1,5 (RLINK) 

n 

P 4- RLINK (P). 

16 

T2C 

J5NZ 

T3 

n 

T2. P = A? 

17 


J6NZ 

T4 

a 

Test if the stack is empty. 

18 

DONE 




1 


Program S. Algorithm S has been augmented with initialization and terinina- 


tion 

conditions to make this program comparable to Program T. 

01 

LLINKT 

EQU 

0:2 



02 

RLINKT 

EQU 

0:2 



03 

SO 

ENT 5 

HEAD 

1 

SO. Initialize. Set P 4— HEAD. 

04 


JMP 

2F 

1 


05 

S3 

JMP 

VISIT 

n 

S3. Visit P. 

06 

SI 

LD5N 

1,5 (RLINKT) 

n 

SI. RLINK (P) a thread? 

07 


J5NN 

IF 

n 

Jump if RTAG(P) = 1. 

08 


ENN6 

0,5 

n — a 

Otherwise set Q 4— RLINK (P) . 

09 

S2 

ENT5 

0,6 

n 

S2. Search to left. Set P 4— 0. 

10 

2H 

LD6 

0,5 (LLINKT) 

n + 1 

Q 4- LLINKT (P). 

11 


J6P 

S2 

n + 1 

If LTAG(P) = 0, repeat. 

12 

1H 

ENT6 

-HEAD, 5 

n + 1 


13 


J6NZ 

S3 

n + 1 

Visit unless P = HEAD. | 
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An analysis of the running time appears with the code above. These quan- 
tities are easy to determine, using Kirchhoff ’s law and the facts that 

i) in Program T, the number of insertions onto the stack must equal the number 
of deletions; 

ii) in Program S, the LLINK and RLINK of each node are examined precisely 
once; 

iii) the number of “visits” is the number of nodes in the tree. 

The analysis tells us Program T takes 15n + a + 4 units of time, and Program S 
takes lln — a + 7 units, where n is the number of nodes in the tree and a is the 
number of terminal right links (nodes with no right subtree). The quantity a 
can be as low as 1, assuming that n/0, and it can be as high as n. If left and 
right are symmetrical, the average value of a is (n + l)/2, as a consequence of 
facts proved in exercise 14. 

The principal conclusions we may reach on the basis of this analysis are: 

i) Step S2 of Algorithm S is performed only once on the average per execution 
of that algorithm, if P is a random node of the tree. 

ii) Traversal is slightly faster for threaded trees, because it requires no stack 
manipulation. 

iii) Algorithm T needs more memory space than Algorithm S because of the 
auxiliary stack required. In Program T we kept the stack in consecutive 
memory locations; therefore we needed to put an arbitrary bound on its 
size. It would be very embarrassing if this bound were exceeded, so it must 
be set reasonably large (see exercise 10); thus the memory requirement 
of Program T is significantly more than Program S. Not infrequently a 
complex computer application will be independently traversing several trees 
at once, and a separate stack will be needed for each tree under Program T. 
This suggests that Program T might use linked allocation for its stack (see 
exercise 20); its execution time then becomes 30n + a + 4 units, roughly 
twice as slow as before, although the traversal speed may not be terribly 
important when the execution time for the other coroutine is added in. Still 
another alternative is to keep the stack links within the tree itself in a tricky 
way, as discussed in exercise 21. 

iv) Algorithm S is, of course, more general than Algorithm T, since it allows us 
to go from P to P$ when we are not necessarily traversing the entire binary 
tree. 

So a threaded binary tree is decidedly superior to an unthreaded one, with 
respect to traversal. These advantages are offset in some applications by the 
slightly increased time needed to insert and delete nodes in a threaded tree. It 
is also sometimes possible to save memory space by “sharing” common subtrees 
with an unthreaded representation, while threaded trees require adherence to a 
strict tree structure with no overlapping of subtrees. 

Thread links can also be used to compute P*, $P, and IIP with efficiency 
comparable to that of Algorithm S. The functions *P and Ptt are slightly harder 
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to compute, just as they are for unthreaded tree representations. The reader is 
urged to work exercise 17. 

Most of the usefulness of threaded trees would disappear if it were hard to 
set up the thread links in the first place. What makes the idea really work is that 
threaded trees grow almost as easily as ordinary ones do. We have the following 
algorithm: 

Algorithm I ( Insertion into a threaded binary tree). This algorithm attaches 
a single node, NODE(Q) , as the right subtree of NODE(P), if the right subtree is 
empty (that is, if RTAG(P) = 1); otherwise it inserts NODE(Q) between NODE(P) 
and NODE(RLINK(P) ) , making the latter node the right child of NODE(Q) . The 
binary tree in which the insertion takes place is assumed to be threaded as in 
( 10 ); for a modification, see exercise 23. 

11. [Adjust tags and links.] Set RLINK(Q) 4 - RLINK(P) , RTAG(Q) RTAG(P) , 
RLINK(P) V- Q. RTAG(P) 4- 0, LLINK(Q) 4- P, LTAG(Q) 4- 1. 

12. [Was RLINK(P) a thread?] If RTAG(Q) = 0, set LLINK(Qs) 4 - Q. (Here Q$ is 
determined by Algorithm S, which will work properly even though LLINK(Qs) 
now points to NODE(P) instead of NODE (Q). This step is necessary only when 
inserting into the midst of a threaded tree instead of merely inserting a new 
leaf. ) | 

By reversing the roles of left and right (in particular, by replacing Qs by $Q 
in step 12), we obtain an algorithm that inserts to the left in a similar way. 

Our discussion of threaded binary trees so far has made use of thread links 
both to the left and to the right. There is an important middle ground between 
the completely unthreaded and completely threaded methods of representation: 
A right-threaded binary tree combines the two approaches by making use of 
threaded RLINKs, while representing empty left subtrees by LLINK = A. (Simi- 
larly, a left-threaded binary tree threads only the null LLINKs.) Algorithm S does 
not make essential use of threaded LLINKs; if we change the test “LTAG = 0” in 
step S2 to "LLINK ^ A”, we obtain an algorithm for traversing right-threaded 
binary trees in symmetric order. Program S works without change in the right- 
threaded case. A great many applications of binary tree structures require only 
a left-to-right traversal of trees using the functions P$ and/or P*, and for these 
applications there is no need to thread the LLINKs. We have described threading 
in both the left and right directions in order to indicate the symmetry and 
possibilities of the situation, but in practice one-sided threading is much more 
common. 

Let us now consider an important property of binary trees, and its con- 
nection to traversal. Two binary trees T and V are said to be similar if they 
have the same structure; formally, this means that (a) they are both empty, or 
(b) they are both nonempty and their left and right subtrees are respectively 
similar. Similarity means, informally, that the diagrams of T and V have the 
same "shape.” Another way to phrase similarity is to say that there is a one-to- 
one correspondence between the nodes of T and T' that preserves the structure: 
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If nodes iq and u 2 in T correspond respectively to u\ and u 2 in T', then ui is in 
the left subtree of u 2 if and only if u[ is in the left subtree of u 2 , and the same 
is true for right subtrees. 

The binary trees T and T' are said to be equivalent if they are similar and if 
corresponding nodes contain the same information. Formally, let info(u) denote 
the information contained in a node u; the trees are equivalent if and only if 
(a) they are both empty, or (b) they are both nonempty and info (root (T)) = 
info(root(T')) and their left and right subtrees are respectively equivalent. 

As examples of these definitions, consider the four binary trees 



in which the first two are dissimilar. The second, third, and fourth are similar 
and, in fact, the second and fourth are equivalent. 

Some computer applications involving tree structures require an algorithm 
to decide whether two binary trees are similar or equivalent. The following 
theorem is useful in this regard: 


Theorem A. Let the nodes of binary trees T and T' be respectively 
ui,u 2 ,...,u n and u' l5 u ' 2 , . . . , u' n , 
in preorder. For any node u let 


l(u) — 1 if u has a nonempty left subtree, 
r( u) = 1 if u has a nonempty right subtree, 

Then T and T' are similar if and only if n = n’ and 


l(u) = 0 otherwise; 
r(u) = 0 otherwise. 


l(uj ) = l(u'j), r (uj) — r(u'j) for 1 < j < n. 
Moreover, T and T' are equivalent if and only if in addition we have 
info (uj) — info(w') for 1 < j < n. 


(n) 

( 12 ) 

(T3) 


Notice that l and r are the complements of the LTAG and RTAG bits in a 
threaded tree. This theorem characterizes any binary tree structure in terms of 
two sequences of Os and Is. 


Proof. It is clear that the condition for equivalence of binary trees will follow 
immediately if we prove the condition for similarity; furthermore the conditions 
n = n' and ( 12 ) are certainly necessary, since corresponding nodes of similar 
trees must have the same position in preorder. Therefore it suffices to prove that 
the conditions ( 12 ) and n = n! are sufficient to guarantee the similarity of T 
and T'. The proof is by induction on n, using the following auxiliary result: 
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Lemma P. Let the nodes of a nonempty binary tree be u \ , u 2 , . . .« , u n in preorder, 
and let f(u) = l(u) + r(u) — 1. Then 

f{ u i) + fi u 2 ) + -'- + f(u n ) = —l, and /(«i) + - • - + /(w/c) > 0, 1 < k < n. (14) 

Proof. The result is clear for n — 1. If n > 1, the binary tree consists of its 
root Ui and further nodes. If /(iq) = 0, then either the left subtree or the right 
subtree is empty, so the condition is obviously true by induction. If /(tq) = 1, 
let the left subtree have n; nodes; by induction we have 

f( u 1) + f f( u k) > 0 fori <k<m, f(ui) + h/(w„,+i) = 0, (15) 

and the condition (14) is again evident. | 

(For other theorems analogous to Lemma P, see the discussion of Polish 
notation in Chapter 10.) 

To complete the proof of Theorem A, we note that the theorem is clearly 
true when n = 0. If n > 0, the definition of preorder implies that tq and u[ are 
the respective roots of their trees, and there are integers and n\ (the sizes of 
the left subtrees) such that 

« 2 > • • • > u n l + 1 an d w 2; • • • 1 U n [+ 1 are the left subtrees of T and T'; 

u n,+2> • • • > u n an d 2 , . . . , u' n are the right subtrees of T and T' . 

The proof by induction will be complete if we can show n, = n\. There are three 
cases: 

if Z(iq) = 0, then n l = 0 = nj; 

if l(ui) = 1, r(tq) = 0, then n t = n — 1 = n'p 

if l(ui) = r(ui ) = 1, then by Lemma P we can find the least k > 0 such 
that f(ui) + ■ ■ ■ + f(iik) = 0; and n l = k - 1 = n\ (see (15)). | 

As a consequence of Theorem A, we can test two threaded binary trees for 
equivalence or similarity by simply traversing them in preorder and checking 
the INFO and TAG fields. Some interesting extensions of Theorem A have been 
obtained by A. J. Blikle, Bull, de l’Acad. Polonaise des Sciences , Serie des 
Sciences Math., Astr., Phys., 14 (1966), 203-208; he considered an infinite class 
of possible traversal orders, only six of which (including preorder) were called 
"addressless” because of their simple properties. 

We conclude this section by giving a typical, yet basic, algorithm for binary 
trees, one that makes a copy of a binary tree into different memory locations. 

Algorithm C ( Copy a binary tree). Let HEAD be the address of the list head 
of a binary tree T; thus, T is the left subtree of HEAD, reached via LLINK(HEAD). 
Let NODE(U) be a node with an empty left subtree. This algorithm makes a copy 
of T and the copy becomes the left subtree of N0DE(U) . In particular, if NODE (U) 
is the list head of an empty binary tree, this algorithm changes the empty tree 
into a copy of T. 

Cl. [Initialize.] Set P HEAD, Q <- U. Go to C4. 
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C2. [Anything to right?] If NODE(P) has a nonempty right subtree, set R <t= 
AVAIL, and attach NODE(R) to the right of NODE(Q). (At the beginning of 
step C2, the right subtree of NODE(Q) was empty.) 

C3. [Copy INFO.] Set INFO(Q) <— INFO(P). (Here INFO denotes all parts of the 
node that are to be copied, except for the links.) 

C4. [Anything to left.?] If NODE(P) has a nonempty left subtree, set R -t= AVAIL, 
and attach NODE(R) to the left of NODE(Q). (At the beginning of step C4, 
the left subtree of NODE(Q) was empty.) 

C5. [Advance.] Set P A- P*, Q <— Q*. 

C6. [Test if complete.] If P = HEAD (or equivalently if Q = RLINK(U), assuming 
that NODE(U) has a nonempty right subtree), the algorithm terminates; 
otherwise go to step C2. | 

This simple algorithm shows a typical application of tree traversal. The 
description here applies to threaded, unthreaded, or partially threaded trees. 
Step C5 requires the calculation of preorder successors P* and Q*; for unthreaded 
trees, this generally is done with an auxiliary stack. A proof of the validity 
of Algorithm C appears in exercise 29; a MIX program corresponding to this 
algorithm in the case of a right-threaded binary tree appears in exercise 2.3.2-13. 
For threaded trees, the “attaching” in steps C2 and C4 is done using Algorithm I. 

The exercises that follow include quite a few topics of interest relating to 
the material of this section. 

Binary or dichotomous systems, although regulated by a principle, 
are among the most artificial arrangements 
that have ever been invented. 

— WILLIAM SWAINSON, A Treatise on the Geography and 

Classification of Animals (1835) 


EXERCISES 


1. [01] In the binary tree ( 2 ), let INFO(P) denote the letter stored in NODE(P). What 
is INFO(LLINK(RLINK(RLINK(T) ) ) ) ? 

2. [11] List the nodes of the binary tree 
order; (c) postorder. 



in (a) preorder; (b) symmetric 


3. [20] Is the following statement true or false? “The terminal nodes of a binary 
tree occur in the same relative position in preorder, inorder, and postorder.” 


► 4. [20] The text defines three basic orders for traversing a binary tree; another 

alternative would be to proceed in three steps as follows: 


a) Visit the root, 

b) traverse the right subtree, 

c) traverse the left subtree, 


using the same rule recursively on all nonempty subtrees. Does this new order bear 
any simple relation to the three orders already discussed? 
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5 . [22] The nodes of a binary tree may be identified by a sequence of zeros and 
ones, in a notation analogous to “Dewey decimal notation” for trees, as follows: The 
root (if present) is represented by the sequence “1”. Roots (if present) of the left 
and right subtrees of the node represented by a are respectively represented by at) 
and al. For example, the node H in (i) would have the representation “1110”. (See 
exercise 2.3-15.) 

Show that preorder, inorder, and postorder can be described conveniently in terms 
of this notation. 

6. [M2 2] Suppose that a binary tree has n nodes that are ui w 2 • • . u n in preorder 
and u pi u P2 . . . u Pn in inorder. Show that the permutation pip 2 ■ ■ ■ p n can be obtained 
by passing 12 ... n through a stack, in the sense of exercise 2.2. 1-2. Conversely, show 
that any permutation pip 2 ■ ■ ■ p n obtainable with a stack corresponds to some binary 
tree in this way. 

7 . [22] Show that if we are given the preorder and the inorder of the nodes of a 
binary tree, the binary tree structure may be constructed. (Assume that the nodes are 
distinct.) Does the same result hold true if we are given the preorder and postorder, 
instead of preorder and inorder? Or if we are given the inorder and postorder? 

8. [20] Find all binary trees whose nodes appear in exactly the same sequence in 
both (a) preorder and inorder; (b) preorder and postorder; (c) inorder and postorder. 
(As in the previous exercise, we assume that the nodes have distinct labels.) 

9 . [M20] When a binary tree having n nodes is traversed using Algorithm T, state 
how many times each of steps Tl, T2, T3, T4, and T5 is performed (as a function of n). 

► 10 . [20] What is the largest number of entries that can be in the stack at once, 
during the execution of Algorithm T, if the binary tree has n nodes? (The answer 
to this question is very important for storage allocation, if the stack is being stored 
consecutively.) 

11 . [HM41] Analyze the average value of the largest stack size occurring during the 
execution of Algorithm T as a function of n, given that all binary trees with n nodes 
are considered equally probable. 

12 . [22] Design an algorithm analogous to Algorithm T that traverses a binary tree 
in preorder, and prove that your algorithm is correct. 

► 13 . [24] Design an algorithm analogous to Algorithm T that traverses a binary tree 
in postorder. 

14 . [22] Show that if a binary tree with n nodes is represented as in ( 2 ), the total 
number of A links in the representation can be expressed as a simple function of n; this 
quantity does not depend on the shape of the tree. 

15 . [15] In a threaded-tree representation like ( 10 ), each node except the list head has 
exactly one link pointing to it from above, namely the link from its parent. Some of the 
nodes also have links pointing to them from below; for example, the node containing C 
has two pointers coming up from below, while node E has just one. Is there any 
simple connection between the number of links pointing to a node and some other 
basic property of that node? (We need to know how many links point to a given node 
when we are changing the tree structure.) 

► 16. [22] The diagrams in Fig. 24 help to provide an intuitive characterization of the 
position ofN0DE(Q$) in a binary tree, in terms of the structure near NQDE(Q) : If NODE(Q) 
has a nonempty right subtree, consider Q = $P, Q$ = P in the upper diagrams; N0DE(Q$) 
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is the “leftmost” node of that right subtree. If NODE(Q) has an empty right subtree, 
consider Q = P in the lower diagrams; N0DE(Q$) is located by proceeding upward in the 
tree until after the first upward step to the right. 

Give a similar “intuitive” rule for finding the position of NODE (Q*) in a binary tree 
in terms of the structure near NODE(Q) . 

► 17. [22] Give an algorithm analogous to Algorithm S for determining P* in a threaded 
binary tree. Assume that the tree has a list head as in (8), (9), and (10). 

18. [24] Many algorithms dealing with trees like to visit each node twice instead of 
once, using a combination of preorder and inorder that we might call double order. 
Traversal of a binary tree in double order is defined as follows: If the binary tree is 
empty, do nothing; otherwise 

a) visit the root, for the first time; 

b) traverse the left subtree, in double order; 

c) visit the root, for the second time; 

d) traverse the right subtree, in double order. 

For example, traversal of (1) in double order gives the sequence 

A\B\D\D2 B 2 A 2 Ci Ei E 2 Gi G 2 C 2 Ei Hi H 2 F 2 J 1 J 2 , 

where A\ means that A is being visited for the first time. 

If P points to a node of the tree and if d = 1 or 2, define (P,d) A = (Q.e) if the 
next step in double order after visiting NODE(P) the dth time is to visit NODE(Q) the eth 
time; or, if (P, d) is the last step in double order, we write (P,d) A = (HEAD, 2), where 
HEAD is the address of the list head. We also define (HEAD, 1) A as the first step in double 
order. 

Design an algorithm analogous to Algorithm T that traverses a binary tree in 
double order, and also design an algorithm analogous to Algorithm S that computes 
(P. d) . Discuss the relation between these algorithms and exercises 12 and 17. 

► 19. [27] Design an algorithm analogous to Algorithm S for the calculation of P# in 
(a) a right-threaded binary tree; (b) a fully threaded binary tree. If possible, the 
average running time of your algorithm should be at most a small constant, when P is 
a random node of the tree. 

20. [23] Modify Program T so that it keeps the stack in a linked list, not in consecutive 
memory locations. 

► 21. [33] Design an algorithm that traverses an unthreaded binary tree in inorder 
without using any auxiliary stack. It is permissible to alter the LLINK and RLINK fields 
of the tree nodes in any manner whatsoever during the traversal, subject only to the 
condition that the binary tree should have the conventional representation illustrated 
in (2) both before and after your algorithm has traversed the tree. No other bits in the 
tree nodes are available for temporary storage. 

22. [25] Write a MIX program for the algorithm given in exercise 21 and compare its 
execution time to Programs S and T. 

23. [22] Design algorithms analogous to Algorithm I for insertion to the right and 
insertion to the left in a right-threaded binary tree. Assume that the nodes have the 
fields LLINK, RLINK, and RTAG. 

24. [M20] Is Theorem A still valid if the nodes of T and T' are given in symmetric 
order instead of preorder? 
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25. [M24] Let T be a set of binary trees in which the value of each info field belongs 
to a given set S , where S is linearly ordered by a relation “A” (see exercise 2.2.3-14). 
Given any trees T, T' in T, let us now define T-<T' if and only if 

i) T is empty; or 

ii) T and T' are not empty, and info(root(T)) -< info(root(T')); or 

iii) T and T' are not empty, info(root(T)) = info(root(T')), left(T) A left(T'), and 
left(T) is not equivalent to left(T'); or 

iv) T and T' are not empty, info(root(T)) = info (root (T 1 )), left(T) is equivalent to 
left(T'), and right(T) A right.(T'). 

Here left(T) and right(T) denote the left and right subtrees of T. Prove that (a) T<T' 
and T' < T" implies T < T"; (b) T is equivalent to T' if and only if T-< T' and T' -< T\ 
(c) for any T, T' in T we have either T A T' or T' ■< T . [Thus, if equivalent trees in T 
are regarded as equal, the relation ■< induces a linear ordering on T. This ordering has 
many applications (for example, in the simplification of algebraic expressions). When 
S has only one element, so that the “info” of each node is the same, we have the special 
case that equivalence is the same as similarity.] 

26. [M24] Consider the ordering TXT' defined in the preceding exercise. Prove 
a theorem analogous to Theorem A, giving a necessary and sufficient condition that 
T< T', and making use of double order as defined in exercise 18. 

► 27. [28] Design an algorithm that tests two given trees T and T' to see whether 
T -< T', T y T', or T is equivalent to T\ in terms of the relation defined in exercise 25, 
assuming that both binary trees are right-threaded. Assume that each node has the 
fields LLINK, RLINK, RTAG, INFO; use no auxiliary stack. 

28. [00] After Algorithm C has been used to make a copy of a tree, is the new binary 
tree equivalent to the original, or similar to it? 

29. [M25] Prove as rigorously as possible that Algorithm C is valid. 

► 30. [22] Design an algorithm that threads an unthreaded tree; for example, it should 
transform ( 2 ) into ( 10 ). Note: Always use notations like P* and P$ when possible, 
instead of repeating the steps for traversal algorithms like Algorithm T. 

31. [23] Design an algorithm that “erases” a right-threaded binary tree. Your algo- 
rithm should return all of the tree nodes except the list head to the AVAIL list, and it 
should make the list head signify an empty binary tree. Assume that each node has 
the fields LLINK, RLINK, RTAG; use no auxiliary stack. 

32. [21 ] Suppose that each node of a binary tree has four link fields: LLINK and RLINK, 
which point to left and right subtrees or A, as in an unthreaded tree; SUC and PRED, 
which point to the successor and predecessor of the node in symmetric order. (Thus 
SUC(P) = P$ and PRED(P) = $P. Such a tree contains more information than a threaded 
tree.) Design an algorithm like Algorithm I for insertion into such a tree. 

► 33. [30] There is more than one way to thread a tree! Consider the following repre- 
sentation, using three fields LTAG, LLINK, RLINK in each node: 

LTAG(P): defined the same as in a threaded binary tree; 

LLINK (P): always equal to P*; 

RLINK(P): defined the same as in an unthreaded binary tree. 

Discuss insertion algorithms for such a representation, and write out the copying 
algorithm, Algorithm C, in detail for this representation. 
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34. [22] Let P point to a node in some binary tree, and let HEAD point to the list head 
of an empty binary tree. Give an algorithm that (i) removes NODE(P) and all of its 
subtrees from whatever tree it was in, and then (ii) attaches NODE(P) and its subtrees 
to NODE (HEAD). Assume that all the binary trees in question are right-threaded, with 
fields LLINK, RTAG, RLINK in each node. 

35. [40] Define a ternary tree (and, more generally, a f-ary tree for any t > 2) in a 
manner analogous to our definition of a binary tree, and explore the topics discussed 
in this section (including topics found in the exercises above) that can be generalized 
to f-ary trees in a meaningful way. 

36. [M23] Exercise 1.2.1-15 shows that lexicographic order extends a well-ordering of 
a set S to a well-ordering of the n-tuples of elements of S. Exercise 25 above shows that 
a linear ordering of the information in tree nodes can be extended to a linear ordering 
of trees, using a similar definition. If the relation -< well-orders S, is the extended 
relation of exercise 25 a well-ordering of T? 

► 37. [24] (D. Ferguson.) If two computer words are necessary to contain two link fields 
and an INFO field, representation ( 2 ) requires 2 n words of memory for a tree with n 
nodes. Design a representation scheme for binary trees that uses less space, assuming 
that one link and an INFO field will fit in a single computer word. 

2.3.2. Binary Tree Representation of Trees 

We turn now from binary trees to just plain trees. Let us recall the basic 
differences between trees and binary trees as we have defined them: 

1) A tree always has a root node, so it is never empty; each node of a tree can 
have 0, 1, 2, 3, ... children. 

2) A binary tree can be empty, and each of its nodes can have 0, 1, or 2 children; 
we distinguish between a “left” child and a “right” child. 

Recall also that a forest is an ordered set of zero or more trees. The subtrees 
immediately below any node of a tree form a forest. 

There is a natural way to represent any forest as a binary tree. Consider 
the following forest of two trees: 



The corresponding binary tree is obtained by linking together the children of 
each family and removing vertical links except from a parent to a first child: 
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Then, tilt the diagram 45° clockwise and tweak it slightly, obtaining a binary 
tree: 


(3) 


Conversely, it is easy to see that any binary tree corresponds to a unique forest 
of trees by reversing the process. 

The transformation from ( 1 ) to ( 3 ) is extremely important; it is called the 
natural correspondence between forests and binary trees. In particular, it gives 
a correspondence between trees and a special class of binary trees, namely the 
binary trees that have a root but no right subtree. (We might also change our 
viewpoint slightly and let the root of a tree correspond to the list head of a binary 
tree, thus obtaining a one-to-one correspondence between trees with n + 1 nodes 
and binary trees with n nodes.) 

Let F = (Ti,T a , . . . ,T„) be a forest of trees. The binary tree B[F) corre- 
sponding to F can be defined rigorously as follows: 

a) If n — 0, B(F) is empty. 

b) If n > 0, the root of B(F) is root(Tj); the left subtree of B(F) is B(Tn, Tin, 
■ ■ ■ ,Ti m ), where Tn, T 12 , . . . , T lm are the subtrees of root(Ti); and the right 
subtree of B(F) is £?(T 2 , . . . , T n ). 

These rules specify the transformation from ( 1 ) to ( 3 ) precisely. 

It will occasionally be convenient to draw our binary tree diagram as in ( 2 ), 
without the 45° rotation. The threaded binary tree corresponding to ( 1 ) is 



r-.. 




<|> \ (&r — KSjprcKe) 




(4) 


(compare with Fig. 24, giving the latter a 45° change in orientation). Notice 
that right thread links go from the rightmost child of a family to the parent. 
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Left thread links do not have such a natural interpretation, due to the lack of 
symmetry between left and right. 

The ideas about traversal explored in the previous section can be recast 
in terms of forests (and, therefore, trees). There is no simple analog of the 
inorder sequence, since there is no obvious place to insert a root among its 
descendants; but preorder and postorder carry over in an obvious manner. Given 
any nonempty forest, the two basic ways to traverse it may be defined as follows: 

Preorder traversal Postorder traversal 

Visit the root of the first tree Traverse the subtrees of the first tree 

Traverse the subtrees of the first tree Visit the root of the first tree 
Traverse the remaining trees Traverse the remaining trees 

In order to understand the significance of these two methods of traversal, 
consider the following notation for expressing tree structure by nested parenthe- 
ses: 

{A(B,C(K)),D(E(H),F(J),G)). (5) 

This notation corresponds to the forest (1): We represent a tree by the informa- 
tion written in its root, followed by a representation of its subtrees; we represent 
a nonempty forest by a parenthesized list of the representations of its trees, 
separated by commas. 

If (1) is traversed in preorder, we visit the nodes in the sequence ABCKD 
E H F J G; this is simply (5) with the parentheses and commas removed. Preorder 
is a natural way to list the nodes of a tree: We list the root first, then the 
descendants. If a tree structure is represented by indentation as in Fig. 20(c), 
the rows appear in preorder. The section numbers of this book itself (see Fig. 22) 
appear in preorder; thus, for example, Section 2.3 is followed by Section 2.3.1. 
then come Sections 2.3.2, 2.3.3, 2.3.4, 2. 3. 4.1, . . . , 2. 3. 4. 6, 2.3.5, 2.4, etc. 

It is interesting to note that preorder is a time-honored concept that might 
meaningfully be called dynastic order. At the death of a king, duke, or earl, the 
title passes to the first son, then to descendants of the first son, and finally if 
these all die out it passes to other sons of the family in the same way. (English 
custom also includes daughters in a family on the same basis as sons, except 
that they come after all the sons.) In theory, we could take a lineal chart of all 
the aristocracy and write out the nodes in preorder; then if we consider only the 
people presently living, we would obtain the order of succession to the throne 
(except as modified by Acts of Abdication). 

Postorder for the nodes in (1) is B K C A H E J F G D\ this is analogous 
to preorder, except that it corresponds to the similar parenthesis notation 

{(B, (. K)C)A , ((H)E, ( J)F , G)D), (6) 

in which a node appears just after its descendants instead of just before. 

The definitions of preorder and postorder mesh very nicely with the natural 
correspondence between trees and binary trees, since the subtrees of the first 
tree correspond to the left binary subtree, and the remaining trees correspond to 
the right binary subtree. By comparing these definitions with the corresponding 
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definitions on page 319, we find that traversing a forest in preorder is exactly 
the same as traversing the corresponding binary tree in preorder. Traversing a 
forest in postorder is exactly the same as traversing the corresponding binary 
tree in inorder. The algorithms developed in Section 2.3.1 may therefore be 
used without change. (Note that postorder for trees corresponds to inorder, 
not postorder, for binary trees. This is fortunate, since we have seen that it 
is comparatively hard to traverse binary trees in postorder.) Because of this 
equivalence, we use the notation P$ for the postorder successor of node P in a 
tree or forest, while it denotes the inorder successor in a binary tree. 

As an example of the application of these methods to a practical problem, 
we will consider the manipulation of algebraic formulas. Such formulas are 
most properly regarded as representations of tree structures, not as one- or two- 
dimensional configurations of symbols, nor even as binary trees. For example, 
the formula y = 31n(x + 1 ) — a/x 2 has the tree representation 



Here the illustration on the left is a conventional tree diagram like Fig. 21, in 
which the binary operators +, — , x, /, and f (the latter denotes exponentiation) 
have two subtrees corresponding to their operands; the unary operator “In” has 
one subtree; variables and constants are terminal nodes. The illustration on the 
right shows the equivalent right-threaded binary tree, including an additional 
node y that is a list head for the tree. The list head has the form described in 
2.3.1 ( 8 ). 

It is important to note that, even though the left-hand tree in ( 7 ) bears 
a superficial resemblance to a binary tree, we are treating it here as a tree , 
and representing it by a quite different binary tree, shown at the right in ( 7 ). 
Although we could develop routines for algebraic manipulations based directly 
on binary tree structures — the so-called “three-address code” representations of 
algebraic formulas — several simplifications occur in practice if we use the general 
tree representation of algebraic formulas, as in ( 7 ), because postorder traversal 
is easier in a tree. 
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The nodes of the left-hand tree in ( 7 ) are 

— X3 1n + xl/a'|' 2:2 in preorder; ( 8 ) 

3a;l + lnxax 2 t/ — in postorder. ( 9 ) 

Algebraic expressions like ( 8 ) and ( 9 ) are very important, and they are known 
as “Polish notations” because form ( 8 ) was introduced by the Polish logician, 
Jan Lukasiewicz. Expression ( 8 ) is the prefix notation for formula ( 7 ), and ( 9 ) 
is the corresponding postfix notation. We will return to the interesting topic of 
Polish notation in later chapters; for now let us be content with the knowledge 
that Polish notation is directly related to the basic orders of tree traversal. 

We shall assume that tree structures for the algebraic formulas with which 
we will be dealing have nodes of the following form in MIX programs: 


RTAG 

RLINK 

TYPE 

LLINK 


IN 

F0 



Here RLINK and LLINK have the usual significance, and RTAG is negative for thread 
links (corresponding to RTAG = 1 in the statements of algorithms). The TYPE 
field is used to distinguish different kinds of nodes: TYPE = 0 means that the node 
represents a constant, and INFO is the value of the constant. TYPE = 1 means 
that the node represents a variable, and INFO is the five-letter alphabetic name 
of this variable. TYPE > 2 means that the node represents an operator; INFO is 
the alphabetic name of the operator and the value TYPE = 2, 3, 4, . . . is used to 
distinguish the different operators +, -, x, /, etc. We will not concern ourselves 
here with how the tree structure has been set up inside the computer memory 
in the first place, since this topic is analyzed in great detail in Chapter 10; let us 
merely assume that the tree already appears in our computer memory, deferring 
questions of input and output until later. 

We shall now discuss the classical example of algebraic manipulation, finding 
the derivative of a formula with respect to the variable x. Programs for algebraic 
differentiation were among the first symbol-manipulation routines ever written 
for computers; they were used as early as 1952. The process of differentiation il- 
lustrates many of the techniques of algebraic manipulation, and it is of significant 
practical value in scientific applications. 

Readers who are not familiar with mathematical calculus may consider this 
problem as an abstract exercise in formula manipulation, defined by the following 


rules: 

D(x) =1 ( 11 ) 

D(a) = 0, if & is a. constant or a variable ^ x ( 12 ) 

D(\nu) =D(u)/u , if u is any formula (13) 

D{-u) — —D(u) ( 14 ) 

D(u + v) = D(u) + D(y) (15) 

D{u - v) - D{u) - D{v) (16) 

D(u x v) = D(u) x v + u x D(v ) (17) 
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D(u/v) = D(u)/v - (u X D(v))/(y t 2 ) (18) 

D(u t v) = D(u) x (y x (u | (v - 1 ))) + ((lnw) x D(v)) x (u f tf) (19) 

These rules allow us to evaluate the derivative D(y) for any formula y composed 
of the operators listed. The ” sign in rule (14) is a unary operator, which 
is different from the binary in (16); we will use “neg” to stand for unary 
negation in the tree nodes below. 

Unfortunately rules (li)-(ig) don’t tell the whole story. If we apply them 
blindly to a rather simple formula like 

y = 3 ln(x + 1 ) — a/x 2 , 

we get 

D{y) = 0 • ln(x + 1 ) + 3((1 + 0 )/(* + 1 )) 

- (0/x 2 - (a(l(2a; 2_1 ) + ((Ins) • 0)x 2 ))/(x 2 ) 2 ), (20) 

which is correct but totally unsatisfactory. To avoid so many redundant opera- 
tions in the answer, we must recognize the special cases of adding or multiplying 
by zero, multiplying by one, or raising to the first power. These simplifications 
reduce (20) to 

D(y) = 3 (l/(s + 1 )) - (( — (a( 2 *)))/(z 2 ) 2 ), ( 21 ) 

which is more acceptable but still not ideal. The concept of a really satisfactory 
answer is not well-defined, because different mathematicians will prefer formulas 
to be expressed in different ways; however, it is clear that (21) is not as simple 
as it could be. In order to make substantial progress over formula (21), it is 
necessary to develop algebraic simplification routines (see exercise 17 ), which 
would reduce (21) to, for example, 

D(y) = 3 (a: + l) -1 + 2 ax~ 3 . (22) 

We will content ourselves here with routines that can produce (21), not (22). 

Our main interest in this algorithm is, as usual, in the details of how the 
process is carried out inside a computer. Many higher-level languages and special 
routines are available at most computer installations, with built-in facilities 
to simplify algebraic manipulations like these; but the purpose of the present 
example is to gain more experience in fundamental tree operations. 

The idea behind the following algorithm is to traverse the tree in postorder, 
forming the derivative of each node as we go, until eventually the entire derivative 
has been calculated. Using postorder means that we will arrive at an operator 
node (like “+”) after its operands have been differentiated. Rules (11) through 
(19) imply that every subformula of the original formula will have to be differen- 
tiated, sooner or later, so we might as well do the differentiations in postorder. 

By using a right-threaded tree, we avoid the need for a stack during the 
operation of the algorithm. On the other hand, a threaded tree representation 
has the disadvantage that we will need to make copies of subtrees; for example, 
in the rule for D(u f v ) we might need to copy u and v three times each. If we 
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had chosen to use a List representation as in Section 2.3.5 instead of a tree, we 
could have avoided such copying. 

Algorithm D ( Differentiation ). If Y is the address of a list head that points to 
a formula represented as described above, and if DY is the address of the list head 
for an empty tree, this algorithm makes NODE(DY) point to a tree representing 
the analytic derivative of Y with respect to the variable “X” . 

Dl. [Initialize.] Set P t— Y$ (namely, the first node of the tree, in postorder. 

which is the first node of the corresponding binary tree in inorder). 

D2. [Differentiate.] Set PI LLINK(P) ; and if PI ^ A, also set Q1 <- RLINK(Pl) . 
Then perform the routine DIFF [TYPE(P)] , described below. (The routines 
DIFF [0] , DIFF [1] , etc., will form the derivative of the tree with root P. and 
will set pointer variable Q to the address of the root of the derivative. The 
variables PI and Q1 are set up first, in order to simplify the specification of 
the DIFF routines.) 

D3. [Restore link.] If TYPE(P) denotes a binary operator, set RLINK(Pl) «- P2. 
(See the next step for an explanation.) 

D4. [Advance to P$.] Set P2 <- P, P <- P$. Now if RTAG(P2) = 0 (that is. if 
N0DECP2) has a sibling to the right), set RLINK(P2) <- Q. (This is the tricky 
part of the algorithm: We temporarily destroy the structure of tree Y. so 
that a link to the derivative of P2 is saved for future use. The missing link 
will be restored later in step D3. See exercise 21 for further discussion of 
this trick.) 

D5. [Done?] If P / Y, return to step D2. Otherwise set LLINK(DY) «- Q and 
RLINK(Q) «- DY, RTAG(Q) <- 1. | 

The procedure described in Algorithm D is just the background routine 
for the differentiation operations that are performed by the processing routines 
DIFF [0] , DIFF [1] , . . . , called in step D2. In many ways, Algorithm D is like the 
control routine for an interpretive system or machine simulator, as discussed in 
Section 1.4.3, but it traverses a tree instead of a simple sequence of instructions. 

To complete Algorithm D we must define the routines that do the actual 
differentiation. In the following discussion, the statement “P points to a tree” 
means that NODE(P) is the root of a tree stored as a right-threaded binary tree, 
although both RLINK(P) and RTAG(P) will be meaningless so far as this tree is 
concerned. We will make use of a tree construction function that makes new 
trees by joining smaller ones together: Let x denote some kind of node, either a 
constant, variable, or operator, and let U and V denote pointers to trees: then 

TREE(x,U,V) makes a new tree with x in its root node and with U and V 
the subtrees of the root: W <t= AVAIL, INFO(W) x, LLINK(W) U. 
RLINK(U) <- V, RTAG(U) <- 0, RLINK(V) «- W, RTAG(V) <- 1. 

TREECrr.U) similarly makes a new tree with only one subtree: W 4= AVAIL. 
INFO (W) <- x, LLINK(W) <- U, RLINK(U) «- W, RTAG(U) y- 1. 

TREE (a:) makes a new tree with x as a terminal root node: W AVAIL. 
INFO (W) <- x, LLINK(W) <- A. 


2.3.2 


BINARY TREE REPRESENTATION OF TREES 341 


Furthermore TYPE(W) is set appropriately, depending on x. In all cases, the 
value of TREE is W, that is, a pointer to the tree just constructed. The reader 
should study these three definitions carefully, since they illustrate the binary 
tree representation of trees. Another function, COPY(U). makes a copy of the 
tree pointed to by U and has as its value a pointer to the tree thereby created. 
The basic functions TREE and COPY make it easy to build up a tree for the 
derivative of a formula, step by step. 

Nullary operators ( constants and variables). For these operations, NODE(P) 
is a terminal node, and the values of PI, P2, Ql, and Q before the operation are 
irrelevant. 

DIFF[0]: (NODE(P) is a constant.) Set Q 4— TREE (0) . 

DIFF [1] : (NODE(P) is a variable.) If INFO(P) = “X”, set Q 4 - TREE ( 1 ) ; 
otherwise set Q 4— TREE(O) . 

Unary operators ( logarithm and negation). For these operations, NODE(P) has 
one child, U, pointed to by PI, and Q points to D(U). The values of P2 and Ql 
before the operation are irrelevant. 

DIFF [2] : (NODE(P) is “In”.) If INFO (Q) /0, set Q4-TREE(“/”,q,C0PY(Pl)). 
DIFF [3] : (NODE(P) is “neg”.) If INFO(Q) / 0, set Q 4- TREE ( “neg” , Q ) . 

Binary operators ( addition , subtraction, multiplication, division, exponentia- 
tion). For these operations, NODE(P) has two children, U and V, pointed to 
respectively by PI and P2; Ql and Q point respectively to D(U), D(V). 

DIFF [4] : (“+” operation.) If INFO(Ql) = 0, set AVAIL 4= Ql. Otherwise if 
INFO(Q) = 0, set AVAIL 4= Q and Q 4— Ql; otherwise set Q 4 — TREE( Ql , Q) . 

DIFF [5]: operation.) If INFO(Q) = 0, set AVAIL 4= Q and Q 4- Ql. 

Otherwise if INF0(Q1) = 0, set AVAIL 4= Ql and set Q 4- T REE ( “neg”, Q) ; 
otherwise set Q 4— TREE( ”, Ql ,Q) . 

DIFF [6] : (“x” operation.) IflNFO(Ql) / 0, set Ql 4— MULTCQl ,C0PY(P2) ) . 
Then if INFO(Q) / 0, set Q 4- MULT(C0PY(P1) ,Q). Then go to DIFF[4], 

Here MULT(U,V) is a new function that constructs a tree for U x V but also 
makes a test to see if U or V is equal to 1 : 

if INFO (U) = 1 and TYPE(U) = 0, set AVAIL 4= U and MULT(U,V) 4- V; 
if INFO (V) = 1 and TYPE(V) = 0, set AVAIL 4= V and MULT(U,V) 4-- U; 
otherwise set MULT(U,V) 4- TREE(“x”,U, V) . 

DIFF [7] : (“/” operation.) If INF0(Q1) / 0, set 

Ql 4- TREE(7”,Q1,C0PY(P2)). 

Then if INF0(Q) 7 0, set 

Q 4- TREE(7”,MULT(C0PY(P1) ,Q) , TREE (“t”, COPY (P2) .TREE (2) ) ) . 

Then go to DIFF [5] . 

DIFF [8]: ( “t” operation.) See exercise 12. 
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We conclude this section by showing how all of the operations above are 
readily transformed into a computer program, starting “from scratch” with only 
MIX machine language as a basis. 

Program D ( Differentiation ). The following MIXAL program performs Algo- 
rithm D, with rI2 = P, rI3 = P2, rI4 = PI, rI5 = Q, rI6 = Ql. The order of 
computations lias been rearranged a little, for convenience. 


001 

* DIFFERENTIATION IN A 

RIGHT-THREADED TREE 

002 

LLINK 

EQU 

4:5 

Definition of fields, see ( 10 ) 

003 

RLINK 

EQU 

1:2 

004 

RLINKT 

EQU 

0:2 


005 

TYPE 

EQU 

3:3 


006 

* MAIN 

CONTROL ROUTINE 

Dl. Initialize. 

007 

D1 

STJ 

9F 

Treat the whole procedure as a subroutine. 

008 


LD4 

Y (LLINK) 

PI 4- LLINK (Y), prepare to find Ys. 

009 

1H 

ENT2 

0,4 

P 4 - PI. 

010 

2H 

LD4 

0 , 2 (LLINK) 

PI 4 - LLINK (P). 

on 


J4NZ 

IB 

If PI ^ A, repeat. 

012 

D2 

LD1 

0,2 (TYPE) 

D2. Differentiate. 

013 


JMP 

*+l , 1 

Jump to DIFF [TYPE(P)] . 

014 


JMP 

CONSTANT 

Switch to table entry for DIFF [0] . 

015 


JMP 

VARIABLE 

DIFF [1] . 

016 


JMP 

LN 

DIFF [2] . 

on 


JMP 

NEG 

DIFF [3], 

018 


JMP 

ADD 

DIFF [4] . 

019 


JMP 

SUB 

DIFF [5], 

020 


JMP 

MUL 

DIFF [6], 

021 


JMP 

DIV 

DIFF [7], 

022 


JMP 

PWR 

DIFF [ 8 ], 

023 

D3 

ST3 

0 , 4(RLINK) 

D3. Restore link. RLINK (P 11 4- P9 

024 

D4 

ENT3 

0,2 

D4. Advance to Ps. P2 4— P. 

025 


LD2 

0,2 (RLINKT) 

P 4 - RLINKT (P) . 

026 


J2N 

IF 

Jump if RTAG(P) = 1; 

027 


ST5 

0,3 (RLINK) 

otherwise set RLINK (P2) 4— Q. 

028 


JMP 

2B 

Note that N0DE(P$) will be terminal. 

029 

1H 

ENN2 

0,2 


030 

D5 

ENT1 

-Y,2 

D5. Done? 

031 


LD4 

0,2 (LLINK) 

PI 4 — LLINK (P), prepare for step D2. 

032 


LD6 

0,4 (RLINK) 

Ql 4 - RLINK(Pl). 

033 


J1NZ 

D2 

Jump to D2 if P / Y; 

034 


ST5 

DY (LLINK) 

otherwise set LLINK (DY) 4— Q. 

035 


ENNA 

DY 

036 


STA 

0,5 (RLINKT) 

RLINK(Q) 4 - DY, RTAG(Q) 4 - 1. 

037 

9H 

JMP 

* 

Exit from differentiation subroutine. | 

The 

next part of the program contains the basic subroutines TREE and COPY. 


former has three entrances TREEO, TREE1, and TREE2, according to the number 
of subtrees of the tree being constructed. Regardless of which entrance to the 
subroutine is used, rA will contain the address of a special constant indicating 
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what type of node forms the root of the tree being constructed; these special 
constants appear in lines 105-124. 


038 

* BASIC 

SUBROUTINES FOR TREE CONSTRUCTION 

039 

TREEO 

STJ 

9F 

TREE(rA) function: 

040 


JMP 

2F 


041 

TREE1 

ST1 

3F (0 : 2) 

TREE(rA,rIl) function: 

042 


JSJ 

IF 


043 

TREE2 

STX 

3F (0 : 2) 

TREE(rA,rX,rIl) function: 

044 

3H 

ST1 

* (RLINKT) 

RLINK(rX) +- rll, RTAG(rX) +- 0. 

045 

1H 

STJ 

9F 


046 


LDXN 

AVAIL 


047 


JXZ 

OVERFLOW 


048 


STX 

0,1 (RLINKT) 

RLINK(rll) +- AVAIL, RTAG(rll) +- 1. 

049 


LDX 

3B(0:2) 


050 


STA 

*+1(0:2) 


051 


STX 

* (LLINK) 

Set LLINK of next root node. 

052 

2H 

LD1 

AVAIL 

rll <= AVAIL. 

053 


J1Z 

OVERFLOW 


054 


LDX 

0,1 (LLINK) 


055 


STX 

AVAIL 


056 


STA 

*+1(0:2) 

Copy root info to new node. 

057 


MOVE 

* (2) 


058 


DEC1 

2 

Reset rll to point to the new root. 

059 

9H 

JMP 

* 

Exit from TREE, rll points to new tree. 

060 

C0PYP1 

ENT1 

0,4 

COPY(Pl), special entrance to COPY 

061 


JSJ 

COPY 


062 

C0PYP2 

ENT1 

0,3 

COPY (P2) , special entrance to COPY 

063 

COPY 

STJ 

9F 

COPY(rll) function: 





(see exercise 13) 

104 

9H 

JMP 

* 

Exit from COPY, rll points to new tree. 

105 

C0N0 

CON 

0 

Node representing the constant “0” 

106 


CON 

0 


107 

CONI 

CON 

0 

Node representing “1” 

108 


CON 

1 


109 

C0N2 

CON 

0 

Node representing “2” 

110 


CON 

2 


111 

LOG 

CON 

2 (TYPE) 

Node representing “In” 

112 


ALF 

LN 


113 

NEGOP 

CON 

3 (TYPE) 

Node representing “neg” 

114 


ALF 

NEG 


115 

PLUS 

CON 

4 (TYPE) 

Node representing “+” 

116 


ALF 

+ 


117 

MINUS 

CON 

5 (TYPE) 

Node representing ” 

118 


ALF 

- 


119 

TIMES 

CON 

6 (TYPE) 

Node representing “x” 

120 


ALF 

* 


121 

SLASH 

CON 

7 (TYPE) 

Node representing “/” 

122 


ALF 

/ 
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123 UP ARROW CON 8 (TYPE) Node representing “4” 

124 ALF ** | 

The remaining portion of the program corresponds to the differentiation routines 
DIFF [0] , DIFF [1] , . . . ; these routines are written to return control to step D3 
after processing a binary operator, otherwise they return to step D4. 


125 * DIFFERENTIATION ROUTINES 

126 VARIABLE 

LDX 1,2 


127 

ENTA CONI 


128 

CMPX 2F 

Is INFO(P) = “X"? 

129 

JE *+2 

If so, call TREE(l) . 

130 CONSTANT 

ENTA C0N0 

Call TREE(0) . 

131 

JMP TREEO 


132 1H 

ENT5 0,1 

0 <r- location of new tree. 

133 

JMP D4 

Return to control routine. 

134 2H 

ALF X 


135 LN 

LDA 1 , 5 


136 

JAZ D4 

Return to control routine if INF0(Q) 

137 

JMP C0PYP1 

otherwise set rll -f- C0PY(P1). 

138 

ENTX 0,5 


139 

ENTA SLASH 


140 

JMP TREE2 

rll <— TREE ( “/” ,Q ,rll) . 

HI 

JMP IB 

Q <— rll, return to control. 

142 NEG 

LDA 1,5 


143 

JAZ D4 

Return if INF0(Q) = 0. 

144 

ENTA NEGOP 


145 

ENT1 0,5 


146 

JMP TREE1 

rll -<- TREE( “neg” , Q) . 

147 

JMP IB 

Q rll, return to control. 

148 ADD 

LDA 1,6 


149 

JANZ IF 

Jump unless INF0(Q1) = (). 

150 3H 

LDA AVAIL 

AVAIL Ql. 

151 

STA 0 , 6 (LLINK) 


152 

ST6 AVAIL 


153 

JMP D3 

Return to control, binary operator. 

154 1H 

LDA 1,5 


155 

JANZ IF 

Jump unless INF0(Q) = 0. 

156 2H 

LDA AVAIL 

AVAIL <= Q. 

157 

STA 0,5 (LLINK) 


158 

ST5 AVAIL 


159 

ENT5 0,6 

Q «- Ql. 

160 

JMP D3 

Return to control. 

161 1H 

ENTA PLUS 

Prepare to call TREE(“+” ,Q1 , Q) . 

162 4H 

ENTX 0,6 


163 

ENT1 0,5 


164 

JMP TREE2 


165 

ENT5 0,1 

Q «- TREE(“±” ,qi,Q). 

166 

JMP D3 

Return to control. 

167 SUB 

LDA 1,5 


168 

JAZ 2B 

Jump if INFO (Q) = 0. 
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169 


LDA 

1,6 

no 


JANZ 

IF 

171 


ENTA 

NEG0P 

172 


ENT1 

0,5 

173 


JMP 

TREE1 

174 


ENT5 

0,1 

175 


JMP 

3B 

176 

1H 

ENTA 

MINUS 

177 


JMP 

4B 

178 

MUL 

LDA 

1,6 

179 


JAZ 

IF 

180 


JMP 

C0PYP2 

181 


ENTA 

0,6 

182 


JMP 

MULT 

183 


ENT6 

0,1 

184 

1H 

LDA 

1,5 

185 


JAZ 

ADD 

186 


JMP 

C0PYP1 

187 


ENTA 

0,1 

188 


ENT1 

0,5 

189 


JMP 

MULT 

190 


ENT5 

0,1 

191 


JMP 

ADD 

192 

MULT 

STJ 

9F 

193 


STA 

IF (0 : 2) 

194 


ST2 

8F (0 : 2) 

195 

1H 

ENT2 

* 

196 


LDA 

1,2 

197 


DECA 

1 

198 


JANZ 

IF 

199 


LDA 

0,2 (TYPE) 

200 


JAZ 

2F 

201 

1H 

LDA 

1,1 

202 


DECA 

1 

203 


JANZ 

IF 

204 


LDA 

0,1 (TYPE) 

205 


JANZ 

IF 

206 


ST1 

*+2(0:2) 

207 


ENT1 

0,2 

208 


ENT2 

♦ 

209 

2H 

LDA 

AVAIL 

210 


STA 

0 , 2 (LLINK) 

211 


ST2 

AVAIL 

212 


JMP 

8F 

213 

1H 

ENTA 

TIMES 

214 


ENTX 

0,2 

215 


JMP 

TREE2 

216 

8H 

ENT2 

* 

217 

9H 

JMP 

* 
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Jump unless INFO(Ql) = 0. 


Q R- TREE(-‘neg” ,Q). 

AVAIL 4 = Q1 and return. 

Prepare to call TREE(“— ” ,Q1 ,Q) . 


Jump if INFO (Ql) = 0; 

otherwise set rll t— C0PY(P2). 

rll +- MULT(Q1,C0PY(P2)). 

Ql -(- rll. 

Jump if INFO(Q) = 0; 

otherwise set rll <— COPY (PI). 


rll +- MULT (COPY (PI) ,Q) . 
Q <- rll. 

MULT(rA,rIl) subroutine: 
Let rA = U, rll = V. 

Save rI2. 
rI2 R- U. 

Test if INFO(U) = 1 


and if TYPE(U) = 0. 

If not, test if INFO(V) = 1 


and if TYPE (V) = 0. 

If so, interchange DRV. 


AVAIL 4 = U. 


Result is V. 


Result is TREE(“x” ,U,V) . 
Restore rI2 setting. 

Exit MULT with result in rll. | 
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The other two routines DIV and PWR are similar and they have been left as 
exercises (see exercises 15 and 16). 


EXERCISES 

► 1. [20] The text gives a formal definition of B(F ), the binary tree corresponding to 
a forest F. Give a formal definition that reverses the process; in other words, define 
F(B), the forest corresponding to a binary tree B. 

► 2. [20] We defined Dewey decimal notation for forests in Section 2.3, and for binary 
trees in exercise 2.3.1 -5. Thus the node “J” in (i) is represented by “2.2.1”, and in 
the equivalent binary tree ( 3 ) it is represented by “11010”. If possible, give a rule 
that directly expresses the natural correspondence between trees and binary trees as a 
correspondence between the Dewey decimal notations. 

3. [22] What is the relation between Dewey decimal notation for the nodes of a 
forest and the preorder and postorder of those nodes? 

4. [19] Is the following statement true or false? “The terminal nodes of a tree occur 
in the same relative position in preorder and postorder.” 

5. [23] Another correspondence between forests and binary trees could be defined by 
letting RLINK(P) point to the rightmost child of NODE (P), and LLINK(P) to the nearest 
sibling on the left. Let F be a forest that corresponds in this way to a binary tree B. 
What order, on the nodes of B, corresponds to (a) preorder (b) postorder on F? 

6. [25] Let T be a nonempty binary tree in which each node has 0 or 2 children. If 
we regard T as an ordinary tree, it corresponds (via the natural correspondence) to 
another binary tree T'. Is there any simple relation between preorder, inorder, and 
postorder of the nodes of T (as defined for binary trees) and the same three orders for 
the nodes of T"! 


7. [M20] A forest may be regarded as a partial ordering, if we say that each node 
precedes its descendants in the tree. Are the nodes topologically sorted (as defined in 
Section 2.2.3) when they are listed in (a) preorder? (b) postorder? (c) reverse preorder? 
(d) reverse postorder? 

8. [M20] Exercise 2.3.1-25 shows how an ordering between the information stored 
in the individual nodes of a binary tree may be extended to a linear ordering of all 
binary trees. The same construction leads to an ordering of all trees, under the natural 
correspondence. Reformulate the definition of that exercise, in terms of trees. 

9. [M21] Show that the total number of nonterminal nodes in a forest has a simple 
relation to the total number of right links equal to A in the corresponding unthreaded 
binary tree. 


10. [M23] Let F be a forest of trees whose nodes in preorder are u\, u 2 , . . . , u n , and 
let F' be a forest whose nodes in preorder are «i, u ' 2 , . . . , u' n ,. Let d(u) denote the 
degree (the number of children) of node u. In terms of these ideas, formulate and prove 
a theorem analogous to Theorem 2.3.1 A. 


11. [15] Draw trees analogous to those shown in (7), corresponding to the formula 
y = e~ x . 

12. [M21] Give specifications for the routine DIFF[8] (the “f’ operation), which was 
omitted from the algorithm in the text. 
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► 13. [26] Write a MIX program for the COPY subroutine (which fits in the program 
of the text between lines 063-104). [Hint: Adapt Algorithm 2.3. 1C to the case of 
right-threaded binary trees, with suitable initial conditions.] 

► 14. [M21] How long does it take the program of exercise 13 to copy a tree with n 
nodes? 

15. [23] Write a MIX program for the DIV routine, corresponding to DIFF [7] as speci- 
fied in the text. (This routine should be added to the program in the text after line 217.) 

16. [24] Write a MIX program for the PWR routine, corresponding to DIFF [8] as speci- 
fied in exercise 12. (This routine should be added to the program in the text after the 
solution to exercise 15.) 

17. [M 40 ] Write a program to do algebraic simplification capable of reducing, for 
example, ( 20 ) or ( 21 ) to ( 22 ). [Hints: Include a new field with each node, representing 
its coefficient (for summands) or its exponent (for factors in a product). Apply algebraic 
identities, like replacing In (u t v) by v In u; remove the operations — , /, f, and neg when 
possible by using equivalent addition or multiplication operations. Make + and X into 
n-ary instead of binary operators; collect like terms by sorting their operands in tree 
order (exercise 8); some sums and products will now reduce to zero or unity, presenting 
perhaps further simplifications. Other adjustments, like replacing a sum of logarithms 
by the logarithm of a product, also suggest themselves.] 

► 18. [25] An oriented tree specified by n links PARENT].)] for 1 < j < n implicitly defines 
an ordered tree if the nodes in each family are ordered by their location. Design an 
efficient algorithm that constructs a doubly linked circular list containing the nodes of 
this ordered tree in preorder. For example, given 

j =12345678 
PARENT [j] = 38404834 

your algorithm should produce 

LLINK[j] = 38467215 
RLINK [)] = 76138452 

and it should also report that the root node is 4. 

19. [M35] A free lattice is a mathematical system, which (for the purposes of this 
exercise) can be simply defined as the set of all formulas composed of variables and 
two abstract binary operators “V” and “A”. A relation “X X Y” is defined between 
certain formulas X and Y in the free lattice by the following rules: 

i) X V Y X W A Z if and only HXVYYW or XVYYZ or XYWAZ or 
yywAZ-, 

ii) X A Y X Z if and only if X X Z and Y X Z; 

hi) X X Y V Z if and only if X X Y and X X Z; 

iv) x X Y A Z if and only if x X Y or x X Z, when a: is a variable; 

v) X V Y X z if and only if X X z or Y X z, when 2 is a variable; 

vi) x X y if and only if x = y, when x and y are variables. 

For example, we find a A (6 V c) X (a A b) V (a A c) >t_ a A [b V c). 

Design an algorithm that tests whether or not X X Y, given two formulas X and Y 
in the free lattice. 
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► 20. [M22] Prove that if u and v are nodes of a forest, u is an ancestor of v if and only 
if u precedes v in preorder and u follows v in postorder. 

21 . [25] Algorithm D controls the differentiation activity for binary operators, unary 
operators, and nullary operators, thus for trees whose nodes have degree 2, 1, and 0; but 
it does not indicate explicitly how the control would be handled for ternary operators 
and nodes of higher degree. (For example, exercise 17 suggests making addition and 
multiplication into operators with any number of operands.) Is it possible to extend 
Algorithm D in a simple way so that it will handle operators of degree more than 2? 

► 22. [ M26 ] If T and T' are trees, let us say T can be embedded in T\ written T C T . 
if there is a one-to-one function / from the nodes of T into the nodes of T' such that 
/ preserves both preorder and postorder. (In other words, u precedes v in preorder 
for T if and only if f(u) precedes f(v) in preorder for T', and the same holds for 
postorder. See Fig. 25.) 



Fig. 25. One tree embedded in another (see exercise 22). 

If T has more than one node, let l[T) be the leftmost subtree of root(T) and let 
r(T) be the rest of T, that is, T with l(T) deleted. Prove that T can be embedded in 
T' if (i) T has just one node, or (ii) both T and T' have more than one node and either 
1 C l(T'), or T C r(T'), or ( l(T ) C l(T') and r(T) C r(T')). Does the converse hold? 

2.3.3. Other Representations of Trees 

There are many ways to represent tree structures inside a computer besides the 
LLINK-RLINK (left child -right sibling) method given in the previous section. 
As usual, the proper choice of representation depends heavily on what kind of 
operations we want to perform on the trees. In this section we will consider a 
few of the tree representation methods that have proved to be especially useful. 

First we can use sequential memory techniques. As in the case of linear lists, 
this mode of allocation is most suitable when we want a compact representation 
of a tree structure that is not going to be subject to radical dynamic changes in 
size or shape during program execution. There are many situations in which we 
need essentially constant tables of tree structures for reference within a program, 
and the desired form of these trees in memory depends on the way in which the 
tables are to be examined. 

The most common sequential representation of trees (and forests) corres- 
ponds essentially to the omission of LLINK fields, by using consecutive addressing 
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instead. For example, let us look again at the forest 


(■ A(B,C(K)),D(E(H),F(J),G )) (i) 


considered in the previous section, which has the tree diagrams 



The preorder sequential representation has the nodes appearing in preorder, 
with the fields INFO, RLINK, and LTAG in each node: 


RLINK j | ^ ^ I ^ 

INFO A D, C K, D E H, F J , G. 

LTAG J J J J J 


(3) 


Here nonnull RLINKs have been indicated by arrows, and LTAG = 1 (for terminal 
nodes) is indicated by “J”. LLINK is unnecessary, since it would either be null 
or it would point to the next item in sequence. It is instructive to compare (1) 
with (3). 

This representation has several interesting properties. In the first place, 
all subtrees of a node appear immediately after that node, so that all subtrees 
within the original forest appear in consecutive blocks. [Compare this with the 
"nested parentheses” in (1) and in Fig. 20 (b).] In the second place, notice that 
the RLINK arrows never cross each other in (3); this will be true in general, for 
in a binary tree all nodes between X and RLINK (X) in preorder lie in the left 
subtree of X, hence no outward arrows will emerge from that part of the tree. 
I11 the third place, we may observe that the LTAG field, which indicates whether 
a node is terminal or not, is redundant, since “J” occurs only at the end of the 
forest and just preceding every downward pointing arrow. 

Indeed, these remarks show that the RLINK field itself is almost redundant; 
all we really need to represent the structure is RTAG and LTAG. Thus it is possible 
to deduce (3) from much less data: 


RTAG 

INFO 

LTAG 




(4) 


As we scan (4) from left to right, the positions with RTAG / correspond 
to nonnull RLINKs that must be filled in. Each time we pass an item with 
LTAG = “J”, we should complete the most recent instance of an incomplete 
RLINK. (The locations of incomplete RLINKs can therefore be kept on a stack.) 
We have essentially proved Theorem 2 . 3 . 1 A again. 

The fact that RLINK or LTAG is redundant in (3) is of little or no help to 
us unless we are scanning the entire forest sequentially, since extra computation 
is required to deduce the missing information. Therefore we often need all of 
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the data in ( 3 ). However, there is evidently some wasted space, since more than 
half of the RLINK fields are equal to A for this particular forest. There are two 
common ways to make use of the wasted space: 

1) Fill the RLINK of each node with the address following the subtree below 
that node. The field is now often called “SCOPE” instead of RLINK, since it 
indicates the right boundary of the “influence” (descendants) of each node. 
Instead of ( 3 ), we would have 


SCOPE 

INFO 


I ll r4l I nff 

ABCKDEHF J G ' 


The arrows still do not cross each other. Furthermore, LTAG(X) = “J” is char- 
acterized by the condition SCOPE (X) = X + c, where c is the number of words per 
node. One example of the use of this SCOPE idea appears in exercise 2.4-12. 

2) Decrease the size of each node by removing the RLINK field, and add 
special "link” nodes just before nodes that formerly had a nonnull RLINK: 


INFO * A * b/c K, D * E H, * F .7, G, ( 6 ) 

LTAQ J J J J J 

Here * indicates the special link nodes, whose INFO somehow characterizes 
them as links pointing as shown by the arrows. If the INFO and RLINK fields of 
(3) occupy roughly the same amount of space, the net effect of the change to (6) 
is to consume less memory, since the number of “*” nodes is always less than 
the number of non- nodes. Representation (6) is somewhat analogous to a 
sequence of instructions in a one-address computer like MIX, with the nodes 
corresponding to conditional jump instructions. 

Another sequential representation analogous to (3) may be devised by omit- 
ting RLINKs instead of LLINKs. In this case we list the nodes of the forest in a new 
order that may be called family order since the members of each family appear 
together. Family order for any forest may be defined recursively as follows: 

Visit the root of the first tree. 

Traverse the remaining trees (in family order). 

Traverse the subtrees of the root of the first tree (in family order). 

(Compare this with the definitions of preorder and postorder in the previous sec- 
tion. Family order is identical with the reverse of postorder in the corresponding 
binary tree.) 

The family order sequential representation of the trees (2) is 


LLINK | | 4 I | } j | | ^ 

INFO A D, E F G, J, H, B C. A', W 

RTAG J J J J j J 

In this case the RTAG entries serve to delimit the families. Family order begins 
by listing the roots of all trees in the forest, then continues by listing individual 
families, successively choosing the family of the most recently appearing node 
whose family has not yet been listed. It follows that the LLINK arrows will 
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never cross; and the other properties of preorder representation carry over in a 
similar way. 

Instead of using family order, we could also simply list the nodes from left 
to right, one level at a time. This is called “level order” [see G. Salton, CACM 5 
(1962), 103 -114], and the level order sequential representation of ( 2 ) is 

LLINK | f=* | }\ | \ ] 

INFO A D. B C. E F G, K, H . J, ( 8 ) 

RTAG J J J J J J 

This is like ( 7 ), but the families are chosen in first-in-first-out fashion rather than 
last-in- first-out. Either ( 7 ) or ( 8 ) may be regarded as a natural analog, for trees, 
of the sequential representation of linear lists. 

The reader will easily see how to design algorithms that traverse and analyze 
trees represented sequentially as above, since the LLINK and RLINK information 
is essentially available just as though we had a fully linked tree structure. 

Another sequential method, called postorder with degrees, is somewhat dif- 
ferent from the techniques above. We list the nodes in postorder and give the 
degree of each node instead of links: 

DEGREE 0012010103 

INFO BKCAHEJFGD ^ 

For a proof that this is sufficient to characterize the tree structure, see exercise 
2.3.2 10. This order is useful for the “bottom-up” evaluation of functions defined 
on the nodes of a tree, as in the following algorithm. 

Algorithm F (Evaluate a locally defined function in a tree). Suppose / is a 
function of the nodes of a tree, such that the value of / at a node x depends only 
on x and the values of / on the children of x. The following algorithm, using an 
auxiliary stack, evaluates / at each node of a nonempty forest. 

FI. [Initialize.] Set the stack empty, and let P point to the first node of the forest 
in postorder. 

F2. [Evaluate /.] Set d t— DEGREE (P) . (The first time this step is reached, d will 
be zero. In general, when we get to this point, it will always be true that 
the top d items of the stack are f(x d ), ..., f(x 1 )— from the top of the 
stack downward — where x x , . . . , x d are the children of N0DE(P) from left 
to right.) Evaluate /(NODE(P)), using the values of f(x d ), . . . , f(x\) found 
on the stack. 

F3. [Update the stack.] Remove the top d items of the stack; then put the value 
/(NODE(P)) on top of the stack. 

F4. [Advance.] If P is the last node in postorder, terminate the algorithm. 
(The stack will then contain /(root(T m )), ..., /(root(Tj)), from top to 
bottom, where T±, ..., T m are the trees of the given forest.) Otherwise 
set P to its successor in postorder (this would be simply P *- P + c in the 
representation ( 9 )), and return to step F2. | 
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The validity of Algorithm F follows by induction on the size of the trees 
processed (see exercise 16). This algorithm bears a striking similarity to the 
differentiation procedure of the previous section (Algorithm 2. 3. 2D), which eval- 
uates a function of a closely related type; see exercise 3. The same idea is used 
in many interpretive routines in connection with the evaluation of arithmetic 
expressions in postfix notation; we will return to this topic in Chapter 8 . See 
also exercise 17, which gives another important procedure similar to Algorithm F. 

Thus we have seen various sequential representations of trees and forests. 
There are also a number of linked forms of representation, which we shall now 
consider. 

The first idea is related to the transformation that takes ( 3 ) into ( 6 ): We 
remove the INFO fields from all nonterminal nodes and put this information as 
a new terminal node below the previous node. For example, the trees ( 2 ) would 
become 



This new form shows that we may assume (without loss of generality) that 
all INFO in a tree structure appears in its terminal nodes. Therefore in the 
natural binary tree representation of Section 2.3.2, the LLINK and INFO fields 
are mutually exclusive and they can share the same field in each node. A node 
might have the fields 


LTAG 


LLINK or INFO 


RLINK 


where the sign LTAG tells whether the second field is a link or not. (Compare 
this representation with, for example, the two-word format of ( 10 ) in Section 
2.3.2.) By cutting INFO down from 5 bytes to 3, we can fit each node into one 
word. However, notice that there are now 15 nodes instead of 10 ; the forest ( 10 ) 
takes 15 words of memory while ( 2 ) takes 20, yet the latter has 50 bytes of INFO 
compared to 30 in the other. There is no real gain in memory space in ( 10 ) 
unless the excess INFO space was going to waste; the LLINKs replaced in ( 10 ) 
are removed at the expense of about the same number of new RLINKs in the 
added nodes. Precise details of the differences between the two representations 
are discussed in exercise 4. 

In the standard binary tree representation of a tree, the LLINK field might 
be more accurately called the LCHILD field, since it points from a parent node to 
its leftmost child. The leftmost child is usually the “youngest” of the children 
in the tree, since it is easier to insert a node at the left of a family than at the 
right; so the abbreviation LCHILD may also be thought of as the “last child” or 
“least child.” 
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Many applications of tree structures require rather frequent references up- 
ward in the tree as well as downward. A threaded tree gives us the ability to 
go upward, but not with great speed; we can sometimes do better if we have a 
third link, PARENT, in each node. This leads to a triply linked tree, where each 
node has LCHILD, RLINK, and PARENT links. Figure 26 shows a triply linked 
tree representation of ( 2 ). For an example of the use of triply linked trees, see 
Section 2.4. 


INFO 

PARENT 

LCHILD 

RLINK 


Fig. 26. A triply linked tree. 



It is clear that the PARENT link all by itself is enough to specify any oriented 
tree (or forest) completely. For we can draw the diagram of the tree if we know all 
the upward links. Every node except the root has just one parent, but there may 
be several children; so it is simpler to give upward links than downward ones. 
Why then haven’t we considered upward links much earlier in our discussion? 
The answer, of course, is that upward links by themselves are hardly adequate 
in most situations, since it is very difficult to tell quickly if a node is terminal 
or not, or to locate any of its children, etc. There is, however, a very important 
application in which upward links are sufficient by themselves: We now turn to 
a brief study of an elegant algorithm for dealing with equivalence relations, due 
to M. J. Fischer and B. A. Galler. 

An equivalence relation “=” is a relation between the elements of a set of 
objects S satisfying the following three properties for any objects x, y, and 2 
(not necessarily distinct) in S: 

i) If x = y and y — z, then x = z. (Transitivity.) 

ii) If x = y, then y = x. (Symmetry.) 

iii) x = x. (R.eflexivity.) 

(Compare this with the definition of a partial ordering relation in Section 2.2.3; 
equivalence relations are quite different from partial orderings, in spite of the fact 
that two of the three defining properties are the same.) Examples of equivalence 
relations are the relation , the relation of congruence (modulo to) for integers, 
the relation of similarity between trees as defined in Section 2.3.1, etc. 
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The equivalence problem is to read in pairs of equivalent elements and to 
determine later whether two particular elements can be proved equivalent or 
not on the basis of the given pairs. For example, suppose that S is the set 
{1, 2, 3, 4, 5, 6 , 7, 8 , 9} and suppose that we are given the pairs 

1 = 5, 6 ee 8 , 7 = 2, 9 = 8 , 3 ee 7, 4 = 2, 9 = 3. (n) 

It follows that, for example, 2 = 6 , since 2 = 7 = 3 = 9 = 8 = 6 . But we cannot 
show that 1 = 6 . In fact, the pairs (n) divide 5 into two classes 

{1,5} and {2, 3, 4, 6 , 7, 8 , 9}, ( 12 ) 

such that two elements are equivalent if and only if they belong to the same 
class. It is not difficult to prove that any equivalence relation partitions its set S 
into disjoint classes (called the equivalence classes), such that two elements are 
equivalent if and only if they belong to the same class. 

Therefore a solution to the equivalence problem is a matter of keeping track 
of equivalence classes like ( 12 ). We may start with each element alone in its 
class, thus: 

{1} {2} {3} {4} {5} { 6 } {7} { 8 } {9} ( 13 ) 

Now if we are given the relation 1 = 5, we put { 1 , 5} together in a class. After 
processing the first three relations 1 = 5, 6 = 8 , and 7 = 2, we will have changed 

(13) to 

{1,5} {2,7} {3} {4} { 6 , 8 } {9}. (r 4 ) 

Now the pair 9 = 8 puts { 6 , 8 , 9} together, etc. 

The problem is to find a good way to represent situations like ( 12 ), ( 13 ), 
and ( 14 ) within a computer so that we can efficiently perform the operations 
of merging classes together and of testing whether two given elements are in 
the same class. The algorithm below uses oriented tree structures for this 
purpose: The elements of S become nodes of an oriented forest; and two nodes 
are equivalent, as a consequence of the equivalent pairs read so far, if and only if 
they belong to the same tree. This test is easy to make, since two elements are in 
the same tree if and only if they are below the same root element. Furthermore, 
it is easy to merge two oriented trees together, by simply attaching one as a new 
subtree of the other’s root. 

Algorithm E ( Process equivalence relations). Let S be the set of numbers 
{1,2, ...,n}, and let PARENT[1], PARENT[2], ..., PARENT[n] be integer vari- 
ables. This algorithm inputs a set of relations such as ( 11 ) and adjusts the PARENT 
table to represent a set of oriented trees, so that two elements are equivalent as 
a consequence of the given relations if and only if they belong to the same 
tree. (Note: In a more general situation, the elements of S would be symbolic 
names instead of simply the numbers from 1 to n; then a search routine, as in 
Chapter 6 , would locate nodes corresponding to the elements of S, and PARENT 
would be a field in each node. The modifications for this more general case are 
straightforward.) 
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El. [Initialize.] Set PARENT [A] 0 for 1 < k < n. (This means that all trees 

initially consist of a root alone, as in (13).) 

E2. [Input new pair.] Get the next pair of equivalent elements “j = A ” from the 
input. If the input is exhausted, the algorithm terminates. 

E3. [Find roots.] If PARENT [j] > 0, set j <- PARENT [j] and repeat this step. 
If PARENT [A] > 0, set k <— PARENT [A-] and repeat this step. (After this 
operation, j and k have moved up to the roots of two trees that are to be 
made equivalent. The input relation j = k was redundant if and only if we 
now have j = k.) 

E4. [Merge trees.] If j ± k, set PARENT [j] <- k. Go back to step E2. | 

The reader should try this algorithm on the input (11). After processing 
1 = 5, 6 = 8, 7 = 2, and 9 = 8, we will have 

PARENT [A]: 500008208 

A: 123456789 

which represents the trees 



After this point, the remaining relations of (11) are somewhat more interesting; 
see exercise 9. 

This equivalence problem arises in many applications. We will discuss 
significant refinements of Algorithm E in Section 7.4.1, when we study the 
connectivity of graphs. A more general version of the problem, which arises when 
a compiler processes “equivalence declarations” in languages like FORTRAN, is 
discussed in exercise 11. 



There are still more ways to represent trees in computer memory. Recall that 
we discussed three principal methods for representing linear lists in Section 2.2: 
the straight representation with terminal link A, the circularly linked lists, and 
the doubly linked lists. The representation of unthreaded binary trees described 
in Section 2.3.1 corresponds to a straight representation in both LLINKs and 


(a) Fields (b) Polynomial = c (constant) (c) Polynomial = go + gi x® 1 + g 2 % 



Fig. 28. Representation of polynomials using four-directional links. Shaded areas of nodes indicate information that is irrelevant in 
the context considered. 
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RLINKs. It is possible to get eight other binary tree representations by indepen- 
dently using any of these three methods in the LLINK and RLINK directions. For 
example, Fig. 27 shows what we get if circular linking is used in both directions. 
If circular links are used throughout as in the figure, we have what is called a 
ring structure ; ring structures have proved to be quite flexible in a number of 
applications. The proper choice of representation depends, as always, on the 
types of insertions, deletions, and traversals that are needed in the algorithms 
that manipulate these structures. A reader who has looked over the examples 
given so far in this chapter should have no difficulty understanding how to deal 
with any of these memory representations. 

We close this section with an example of modified doubly linked ring struc- 
tures applied to a problem we have considered before: arithmetic on polynomials. 
Algorithm 2.2.4A performs the addition of one polynomial to another, given that 
the two polynomials are expressed as circular lists; various other algorithms in 
that section give other operations on polynomials. However, the polynomials 
of Section 2.2.4 are restricted to at most three variables. When multi-variable 
polynomials are involved, it is usually more appropriate to use a tree structure 
instead of a linear list. 

A polynomial either is a constant or has the form 


5Z 9j xH t 

0<j<n 


where a: is a variable, n > 0, 0 = e 0 < e x < ■ • • < e„, and g 0 ,...,g n are 
polynomials involving only variables alphabetically less than x\ gi , . . . , g n are not 
zero. This recursive definition of polynomials lends itself to tree representation 
as indicated in Fig. 28. Nodes have six fields, which in the case of MIX might fit 
in three words: 


+ 

0 

LEFT 

RIGHT 

+ 

EXP 

UP 

DOWN 



CV 



(i7) 


Here LEFT, RIGHT. UP, and DOWN are links; EXP is an integer representing an 
exponent; and CV is either a constant (coefficient) or the alphabetic name of a 
variable. The root node has UP = A, EXP = 0, LEFT = RIGHT = * (self). 

The following algorithm illustrates traversal, insertion, and deletion in such 
a four-way-linked tree, so it bears careful study. 


Algorithm A ( Addition of polynomials). This algorithm adds polynomial (P) to 
polynomial ( Q) , assuming that P and Q are pointer variables that link to the roots 
of distinct polynomial trees having the form shown in Fig. 28. At the conclusion 
of the algorithm, polynomial (P) will be unchanged, and polynomial (Q) will 
contain the sum. 

Al. [Test type of polynomial.] If DOWN(P) = A (that is, if P points to a 
constant), then set Q <— DOWN(Q) zero or more times until DOWN(Q) = A 
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and go to A3. If DOWN (P) ^ A, then if DOWN (Q) = A or if CV(Q) < CV(P), 
go to A2. Otherwise if CV(Q) = CV(P), set P 4- DOWN(P), Q 4 - DOWN(Q) 
and repeat this step; if CV(Q) > CV(P), set Q 4- DOWN(Q) and repeat this 
step. (Step A1 either finds two matching terms of the polynomials or 
else determines that an insertion of a new variable must be made into the 
current part of polynomial (Q) .) 

A2. [Downward insertion.] Set R AVAIL, S 4- DOWN(Q). If S / A. set 
UP(S) 4 R, S 4 RIGHT(S) and if EXP(S) ^ 0, repeat this operation 
until ultimately EXP(S) = 0. Set UP(R) 4- Q, DOWN(R) 4- DOWN(Q), 
LEFT (R) 4- R, RIGHT (R) 4- R, CV(R) 4- CV(Q) and EXP(R) 4- 0. Finally, 
set CV (Q) 4— CV (P) and DOWN (Q) 4- R, and return to Al. (We have inserted 
a “dummy” zero polynomial just below NODE(Q) , to obtain a match with 
a corresponding polynomial found within P’s tree. The link manipulations 
done in this step are straightforward and may be derived easily using 
“before-and-after” diagrams, as explained in Section 2.2.3.) 

A3. [Match found.] (At this point, P and Q point to corresponding terms of 
the given polynomials, so addition is ready to proceed.) Set CV(Q) 4 - 
CV(Q) + CV(P). If this sum is zero and if EXP(Q) ^ 0, go to step A8. If 
EXP(Q) = 0, go to A7. 

A4. [Advance to left.] (After successfully adding a term, we look for the next 
term to add.) Set P 4— LEFT(P). If EXP(P) = 0, go to A6. Otherwise 
set Q 4- LEFT (Q) one or more times until EXP(Q) < EXP(P). If then 
EXP (Q) = EXP(P), return to step Al. 

A5. [Insert to right.] Set R AVAIL. Set UP(R) 4- UP(Q), D0WN(R) 4- A, 
CV(R) 4- 0, LEFT(R) 4- Q, RIGHT (R) 4- RIGHT(Q) , LEFT (RIGHT (R) ) 4- R, 
RIGHT(Q) 4— R, EXP(R) 4— EXP(P), and Q 4— R. Return to step Al. (We 
needed to insert a new term in the current row, just to the right of N0DE(Q) , 
in order to match a corresponding exponent in polynomial (P) . As in 
step A2, a “before-and-after” diagram makes the operations clear.) 

A6. [Return upward.] (A row of polynomial (P) has now been completely tra- 
versed.) Set P 4— UP(P). 

A7. [Move Q up to right level.] If UP(P) = A, go to All; otherwise set 
q 4— UP(q) zero or more times until CV(UP(q)) = CV(UP(P)). Return 
to step A4. 

A8. [Delete zero term.] Set R 4- q, q 4- RIGHT (R) , S 4- LEFT(R) , LEFT(q) 4 - S. 
RIGHT (S) 4— q, and AVAIL <t= R. (Cancellation occurred, so a row element 
of polynomial (q) is deleted.) If now EXP(LEFT(P)) = 0 and q = S, go 
to A9; otherwise return to A4. 

A9. [Delete constant polynomial.] (Cancellation has caused a polynomial to 
reduce to a constant, so a row of polynomial(q) is deleted.) Set R 4 — q, 
q 4- UPCq), D0WN(q) 4- D0WN(R), CV(q) 4- CV(R), and AVAIL 4 = R. Set 
S 4- DOWN (q) ; if S ^ A, set UP(S) 4- q, S 4- RIGHT (S) , and if EXP(S) / 0, 
repeat this operation until ultimately EXP(S) — 0. 
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A10. [Zero detected?] If DOWN(Q) = A, CV(Q) = 0, and EXP(Q) ^ 0, set P «- 
UP(P) and go to A8; otherwise go to A6. 

All. [Terminate.] Set Q 4— UP(Q) zero or more times until UP(Q) = A (thus 
bringing Q to the root of the tree). | 

This algorithm will actually run much faster than Algorithm 2.2.4A if poly- 
nomial(P) has few terms and polynomial(Q) has many, since it is not necessary 
to pass over all of polynomial (Q) during the addition process. The reader will 
find it instructive to simulate Algorithm A by hand, adding the polynomial 
xy — x 2 — xyz — z 3 + 3 xz 3 to the polynomial shown in Fig. 28. (This case does 
not demonstrate the efficiency of the algorithm, but it makes the algorithm go 
through all of its paces by showing the difficult situations that must be handled.) 
For further commentary on Algorithm A, see exercises 12 and 13. 

No claim is being made here that the representation shown in Fig. 28 is the 
"best” for polynomials in several variables; in Chapter 8 we will consider another 
format for polynomial representation, together with arithmetic algorithms using 
an auxiliary stack, with significant advantages of conceptual simplicity when 
compared to Algorithm A. Our main interest in Algorithm A is the way it 
typifies manipulations on trees with many links. 

EXERCISES 

1. [20] If we had only LTAG, INFO, and RTAG fields (not LLINK) in a level order 
sequential representation like ( 8 ), would it be possible to reconstruct the LLINKs? (In 
other words, are the LLINKs redundant in ( 8 ), as the RLINKs are in ( 3 )?) 

2 . [22] (Burks, Warren, and Wright, Math. Comp. 8 (1954), 53-57.) The trees ( 2 ) 
stored in preorder with degrees would be 

DEGREE 2 0 1 0 3 1 0 1 0 0 

INFO ABC KDEHFJG 

[compare with ( 9 ), where postorder was used]. Design an algorithm analogous to 
Algorithm F to evaluate a locally defined function of the nodes by going from right to 
left in this representation. 

3. [24] Modify Algorithm 2.3. 2D so that it follows the ideas of Algorithm F, placing 
the derivatives it computes as intermediate results on a stack, instead of recording their 
locations in an anomalous fashion as is done in step D3. (See exercise 2.3.2-21.) The 
stack may be maintained by using the RLINK field in the root of each derivative. 

4. [18] The trees ( 2 ) contain 10 nodes, five of which are terminal. Representation 
of these trees in the normal binary-tree fashion involves 10 LLINK fields and 10 RLINK 
fields (one for each node). Representation of these trees in the form ( 10 ), where LLINK 
and INFO share the same space in a node, requires 5 LLINKs and 15 RLINKs. There are 
10 INFO fields in each case. 

Given a forest with n nodes, m of which are terminal, compare the total number of 
LLINKs and RLINKs that must be stored using these two methods of tree representation. 

5. [16] A triply linked tree, as shown in Fig. 26, contains PARENT, LCHILD, and RLINK 
fields in each node, with liberal use of A-links when there is no appropriate node to 
mention in the PARENT, LCHILD, or RLINK field. Would it be a good idea to extend this 
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representation to a threaded tree, by putting “thread” links in place of the null LCHILD 
and RLINK entries, as we did in Section 2.3.1? 

► 6. [24] Suppose that the nodes of an oriented forest have three link fields, PARENT. 
LCHILD, and RLINK, but only the PARENT link has been set up to indicate the tree 
structure. The LCHILD field of each node is A and the RLINK fields are set as a linear 
list that simply links the nodes together in some order. The link variable FIRST points 
to the first node, and the last node has RLINK = A. 

Design an algorithm that goes through these nodes and fills in the LCHILD and 
RLINK fields compatible with the PARENT links, so that a triply linked tree representation 
like that in Fig. 26 is obtained. Also, reset FIRST so that it now points to the root of 
the first tree in this representation. 

7. [15] What classes would appear in (12) if the relation 9 = 3 had not been given 
in (11)? 

8. [15] Algorithm E sets up a tree structure that represents the given pairs of equiv- 
alent elements, but the text does not mention explicitly how the result of Algorithm E 
can be used. Design an algorithm that answers the question, “Is j = /,■?” . assuming 
that 1 < j < n, 1 < k < n, and that Algorithm E has set up the PARENT table for some 
set of equivalences. 

9. [20] Give a table analogous to (15) and a diagram analogous to (16) that shows 
the trees present after Algorithm E has processed all of the equivalences in (11) from 
left to right. 

10 . [28] In the worst case, Algorithm E may take order n 2 steps to process n equiv- 
alences. Show how to modify the algorithm so that the worst case is not this bad. 

► 11 . [24] ( Equivalence declarations.) Several compiler languages, notably FORTRAN, 
provide a facility for overlapping the memory locations assigned to sequentially stored 
tables. The programmer gives the compiler a set of relations of the form X[j] = 
Y [fc] , which means that variable X [ j + s] is to be assigned to the same location as 
variable Y[A T s] for all s. Each variable is also given a range of allowable subscripts: 

ARRAY X[/:u]’ means that space is to be set aside in memory for the table entries 
X[Z],X[Z + 1], . . . , X[u] . For each equivalence class of variables, the compiler reserves 
as small a block of consecutive memory locations as possible, to contain all the table 
entries for the allowable subscript values of these variables. 

For example, suppose we have ARRAY X [0: 10] , ARRAY Y [3: 10] , ARRAY A [1:1], and 
ARRAY Z [—2:0] , plus the equivalences X[7] S Y [3] , Z[0] = A [0] , and Y [1] = A [8] . 
We must set aside 20 consecutive locations 

Xo Xi Xo X3 X4 X5 X6 X7 X 8 X9 X10 


Z^Z-J Zo Ai Y 3 Y 4 Y 5 Y 6 Y 7 Y s Y 9 Y 10 

for these variables. (The location following A [1] is not an allowable subscript value for 
any of the arrays, but it must be reserved anyway.) 

The object of this exercise is to modify Algorithm E so that it applies to the 
more general situation just described. Assume that we are writing a compiler for 
such a language, and the tables inside our compiler program itself have one node for 
each array, containing the fields NAME, PARENT, DELTA, LBD, and UBD. Assume that 
the compiler program has previously processed all the ARRAY declarations, so that if 
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ARRAY X [/ : u] has appeared and if P points to the node for X, then 
NAME(P) = “X”, PARENT (P) = A, DELTA (P) = 0, 

LBD(P) = Z, UBD(P) = u. 

The problem is to design an algorithm that processes the equivalence declarations, so 
that, after this algorithm has been performed, 

PARENT(P) = A means that locations X[LBD(P)] , . . . ,X[UBD(P)] are to be reserved 
in memory for this equivalence class; 

PARENT(P) = Q / A means that location X[fc] equals location Ylk + DELTA(P)] , 
where NAME(Q) = “Y” . 

For example, before the equivalences listed above we might have the nodes 


p 

NAHE(P) 

PARENT (P) 

DELTA (P) 

LBD(P) 

UBD(P) 

a 

X 

A 

0 

0 

10 

P 

Y 

A 

0 

3 

10 

7 

A 

A 

0 

1 

1 

<5 

Z 

A 

0 

-2 

0 

After the 

equivalences 

are processed, 

the nodes 

might appear 

thus: 

a 

X 

A 

* 

-5 

14 

P 

Y 

a 

4 

* 

* 

7 

A 

6 

0 

* 

* 

<5 

Z 

a 

-3 

* 

* 


(“*” denotes irrelevant information.) 

Design an algorithm that makes this transformation. Assume that inputs to your 
algorithm have the form (P,j, Q,fc), denoting X[j] ~ Y[k], where NAME(P) = “X” and 
NAME(Q) = “Y”. Be sure to check whether the equivalences are contradictory; for 
example, X [1] = Y[2] contradicts X[2] = Y [1] . 

12. [21] At the beginning of Algorithm A, the variables P and Q point to the roots of 
two trees. Let Po and Q 0 denote the values of P and Q before execution of Algorithm A. 
(a) After the algorithm terminates, is Q 0 always the address of the root of the sum of 
the two given polynomials? (b) After the algorithm terminates, have P and Q returned 
to their original values Po and Q 0 ? 

► 13. [ M29 ] Give an informal proof that at the beginning of step A 8 of Algorithm A we 
always have EXP(P) = EXP(Q) and CV(UP(P)) = CV(UP(Q)). (This fact is important to 
the proper understanding of that algorithm.) 

14. [40] Give a formal proof (or disproof) of the validity of Algorithm A. 

15. [40] Design an algorithm to compute the product of two polynomials represented 
as in Fig. 28. 

16. [M24] Prove the validity of Algorithm F. 

► 17. [25] Algorithm F evaluates a “bottom-up” locally defined function, namely, one 
that should be evaluated at the children of a node before it is evaluated at the node. 
A “top-down” locally defined function / is one in which the value of / at a node x 
depends only on x and the value of f at the parent of x. Using an auxiliary stack, 
design an algorithm analogous to Algorithm F that evaluates a “top-down” function / 
at each node of a tree. (Like Algorithm F, your algorithm should work efficiently on 
trees that have been stored in postorder with degrees, as in ( 9 ).) 


362 INFORMATION STRUCTURES 


2.3.3 


P 




► 18. [28] Design an algorithm that, given the two tables INFOlCj] and RLINK [ j] for 
1 < j < n corresponding to preorder sequential representation, forms tables INF02[j] 
and DEGREE [j] for 1 < j < n, corresponding to postorder with degrees. For example, 
according to (3) and (9), your algorithm should transform 

j 123456789 10 

INF01 [j] ABCKDEHFJG 

RLINK [j] 5300080 10 00 

into 

INF02 [j] BKCAHEJFGD 

DEGREE [j] 0012010103 

19. [M27] Instead of using SCOPE links in (5), we could simply list the number of 
descendants of each node, in preorder: 

DESC 3010510100 

INFO ABCKDEHFJG 

Let did 2 . . . d n be the sequence of descendant numbers of a forest, obtained in this way. 

a) Show that k + d k < n for 1 < k < n, and that k < j < k+d k implies j+dj < k + d k . 

b) Conversely, prove that if did 2 ...d n is a sequence of nonnegative integers satisfying 
the conditions of (a), it is the sequence of descendant numbers of a forest. 

c) Suppose d 1 d 2 . . . d n and . . . d' n are the descendant number sequences for two 
forests. Prove that there is a third forest whose descendant numbers are 

min(d 1 , dj ) min(d 2 , d' 2 ) . . . min(d n , d ' n ) . 

2.3.4. Basic Mathematical Properties of Trees 

Tree structures have been the object of extensive mathematical investigations for 
many years, long before the advent of computers, and many interesting facts have 
been discovered about them. In this section we will survey the mathematical 
theory of trees, which not only gives us more insight into the nature of tree 
structures but also has important applications to computer algorithms. 

Nonmat hematical readers are advised to skip to subsection 2. 3. 4. 5, which 
discusses several topics that arise frequently in the applications we shall study 
later. 

The material that follows comes mostly from a larger area of mathematics 
known as the theory of graphs. Unfortunately, there will probably never be 
a standard terminology in this field, and so the author has followed the usual 
practice of contemporary books on graph theory, namely to use words that are 
similar but not identical to the terms used in any other books on graph theory. 
An attempt has been made in the following subsections (and, indeed, throughout 
this book) to choose short, descriptive words for the important concepts, selected 
from those that are in reasonably common use and that do not sharply conflict 
with other common terminology. The nomenclature used here is also biased 
towards computer applications. Thus, an electrical engineer may prefer to call a 
“tree” what we call a “free tree” ; but we want the shorter term “tree” to stand for 
the concept that is generally used in the computer literature and that is so much 
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Fig. 30. A free tree. 

more important in computer applications. If we were to follow the terminology 
of some authors on graph theory, we would have to say “finite labeled rooted 
ordered tree” instead of just “tree,” and “topological bifurcating arborescence” 
instead of “binary tree” ! 

2. 3. 4.1. Free trees. A graph is generally defined to be a set of points (called 
vertices ) together with a set of lines (called edges) joining certain pairs of distinct 
vertices. There is at most one edge joining any pair of vertices. Two vertices 
are called adjacent if there is an edge joining them. If V and V are vertices 
and if n > 0, we say that (Vo, Vj, . . . , V n ) is a path of length n from V to V' if 
V ~ Vb, Vk is adjacent to Vk+i for 0 < k < n, and V n = V'. The path is simple 
if Vo, V j , . . . , F n _i are distinct and if Vi, . . . , V n -i, V n are distinct. A graph is 
connected if there is a path between any two vertices of the graph. A cycle is a 
simple path of length three or more from a vertex to itself. 

These definitions are illustrated in Fig. 29, which shows a connected graph 
with five vertices and six edges. Vertex C is adjacent to A but not to B; there 
are two paths of length two from B to C, namely (B, A , C) and (B, D, C). There 
are several cycles, including (B, D, E, B). 

A free tree or “unrooted tree” (Fig. 30) is defined to be a connected graph 
with no cycles. This definition applies to infinite graphs as well as to finite ones, 
although for computer applications we naturally are most concerned with finite 
trees. There are many equivalent ways to define a free tree; some of them appear 
in the following well-known theorem: 

Theorem A. If G is a graph, the following statements are equivalent: 

a) G is a free tree. 

b) G is connected, but if any edge is deleted, the resulting graph is no longer 
connected. 

c) IfV and V are distinct vertices of G, there is exactly one simple path from 
V to V'. 

Furthermore, if G is finite, containing exactly n > 0 vertices, the following 
statements are also equivalent to (a), (b), and (c): 

d) G contains no cycles and has n — 1 edges. 

e) G is connected and has n — 1 edges. 


B 



• E 


Fig. 29. A graph. 
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Proof, (a) implies (b), for if the edge V — V' is deleted but G is still connected, 
there must be a simple path (V, V lt . . , , V') of length two or more — see exer- 
cise 2 — and then ( V , V u ..., V', V) would be a cycle in G. 

(b) implies (c), for there is at least one simple path from V to V. And if 
there were two such paths (V, V u , . . , V') and (V, V/, . . . , V' ), we could find the 
smallest k for which V k / V k ; deleting the edge V fc _! — V k would not disconnect 
the graph, since there would still be a path {V k _ i , V k , ... ,V .... , V k ) from V k _ 1 
to Vk that does not use the deleted edge. 

(c) implies (a), for if G contains a cycle (V,V\,..., V), there are two simple 
paths from V to Vi. 

To show that (d) and (e) are also equivalent to (a), (b), and (c), let us first 
prove an auxiliary result: If G is any finite graph that has no cycles and at least 
one edge, then there is at least one vertex that is adjacent to exactly one other 
vertex. This follows because we can find some vertex Vi and an adjacent vertex 
V 2 , for k > 2 either V k is adjacent to Vj._ 1 and no other, or it is adjacent to a 
vertex that we may call Vj. +1 -f V k ~i- Since there are no cycles, Vj, V 2 , . . . , V fc+1 
must be distinct vertices, so this process must ultimately terminate. 

Now assume that G is a tree with n > 1 vertices, and let V n be a vertex that 
is adjacent to only one other vertex, namely V n _ x . If we delete V n and the edge 
Vri-i Ki, the remaining graph G' is a tree, since V n appears in 110 simple path 
of G except as the first or the last element. This argument proves (by induction 
on n) that G has n — 1 edges; hence (a) implies (d). 

Assume that G satisfies (d) and let V n , & be as in the preceding 

paragraph. Then the graph G is connected, since V n is connected to V n _ , . 
which (by induction on n) is connected to all other vertices of G 1 . Thus (d) 
implies (e). 

Finally assume that G satisfies (e). If G contains a cycle, we can delete any 
edge appearing in that cycle and G would still be connected. We can therefore 
continue deleting edges in this way until we obtain a connected graph G’ with 
n ~ l - k edges and no cycles. But since (a) implies (d), we must have k = 0. 
that is, G = G'. | 

The idea of a free tree can be applied directly to the analysis of computer 
algorithms. In Section 1.3.3, we discussed the application of Kirchhoff’s first 
law to the problem of counting the number of times each step of an algorithm 
is performed; we found that Kirchhoff’s law does not completely determine the 
number of times each step is executed, but it reduces the number of unknowns 
that must be specially interpreted. The theory of trees tells us how many 
independent unknowns will remain, and it gives us a systematic way to find them. 

It is easier to understand the method that follows if an example is studied, 
so we will work an example as the theory is being developed. Figure 31 shows an 
abstracted flow chart for Program 1.3.3A, which was subjected to a “Kirchhoff's 
law analysis in Section 1.3.3. Each box in Fig. 31 represents part of the 
computation, and the letter or number inside the box denotes the number of 
times that computation will be performed during one run of the program, using 
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Fig. 31. Abstracted flow chart of Program 1.3.3A. 


the notation of Section 1.3.3. An arrow between boxes represents a possible 
jump in the program. The arrows have been labeled e 1; e 2 , . . . , e 2 7 . Our goal is 
to And all relations between the quantities A , B, C , D , E, F, G , H , J, K , L, P, 
Q, R, and S that are implied by Kirchhoff’s law, and at the same time we hope 
to gain some insight into the general problem. (Note: Some simplifications have 
already been made in Fig. 31; for example, the box between C and E has been 
labeled “ 1 ”, and this in fact is a consequence of Kirchhoff’s law.) 

Let Ej denote the number of times branch e,- is taken during the execution 
of the program being studied; Kirchhoff’s law is 

sum of E's into box = value in box = sum of E's leaving box; (i) 

for example, in the case of the box marked K we have 

£19 + E 20 = K — Eyg + E 21 • ( 2 ) 

In the discussion that follows, we will regard E\,E 2 , . . . , E 27 as the unknowns, 
instead of A, B S . 

The flow chart in Fig. 31 may be abstracted further so that it becomes 
a graph G as in Fig. 32. The boxes have shrunk to vertices, and the arrows 
ei.e. 2 , ■ ■ ■ now represent edges of the graph. (A graph, strictly speaking, has 
no implied direction in its edges, and the direction of the arrows should be 
ignored when we refer to graph-theoretical properties of G. Our application 
to Kirchhoff’s law, however, makes use of the arrows, as we will see shortly.) 
For convenience an extra edge e 0 has been drawn from the Stop vertex to the 
Start vertex, so that Kirchhoff’s law applies uniformly to all parts of the graph. 
Figure 32 also includes some other minor changes from Fig. 31: An extra vertex 
and edge have been added to divide e 13 into two parts e ' 13 and e" 3 , so that the 
basic definition of a graph (no two edges join the same two vertices) is valid; e 19 
has also been split up in this way. A similar modification would have been made 
if we had any vertex with an arrow leading back to itself. 

Some of the edges in Fig. 32 have been drawn much heavier than the others. 
These edges form a free subtree of the graph, connecting all the vertices. It 
is always possible to find a free subtree of the graphs arising from flow charts, 
because the graphs must be connected and, by part (b) of Theorem A, if G is 
connected and not a free tree, we can delete some edge and still have the resulting 
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Fig. 32. Graph corresponding to Fig. 31, including a free subtree. 


graph connected; this process can be iterated until we reach a subtree. Another 
algorithm for finding a free subtree appears in exercise 6. We can in fact always 
discard the edge e 0 (which went from the Stop to the Start vertex) first; thus we 
may assume that eo does not appear in the subtree chosen. 

Let G' be a free subtree of the graph G found in this way, and consider 
any edge V V' of G that is not in G'. We may now note an important 
consequence of Theorem A: G' plus this new edge V — V ' contains a cycle; and 
in fact there is exactly one cycle, having the form ( V. V f , . . . , V^), since there is 
a unique simple path from V to V in G'. For example, if G' is the free subtree 
shown in Fig. 32 , and if we add the edge e2, we obtain a cycle that goes along 
e 2 and then (in the direction opposite to the arrows) along e 4 and e 3 . This cycle 
may be written algebraically as “e 2 —64 — 63”, using plus signs and minus signs 
to indicate whether the cycle goes in the direction of the arrows or not. 

If we carry out this process for each edge not in the free subtree, we obtain 
the so-called fundamental cycles, which in the case of Fig. 32 are 

Go: e 0 + ei + e 3 + e 4 + e 6 + e7 + e 9 + ei 0 + e n + e 12 + e 14 , 

G 2 : e 2 — e 4 — e 3 , 

G5: es — 67 — e 6 , 

Cs'- es + e 3 -(- e 4 + e6 + 67, 

Cl3 : e 13 + e 12 + e 'i3, 

C17: ei7 -)- e 22 + e 24 + e 27 + en + e 15 + e 16 , 
e" 9 + e 18 + e' 19 , 

G 2 o: e 2 o + e 4 8 + e 22 + e 23 , 

C2U e2i - ej 6 - e 15 - en - e 2 7 - e 24 - e 22 - ei 8 , 

G 2 5: e 2 5 + e 2 6 — e 2 7. 
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Obviously an edge ej that is not in the free subtree will appear in only one of 
the fundamental cycles, namely Cj. 

We are now approaching the climax of this construction. Each fundamental 
cycle represents a solution to Kirchhoff’s equations; for example, the solution 
corresponding to C 2 is to let E 2 = +1, E 4 = - 1 , E 3 = - 1 , and all other £”s = 0. 
It is clear that flow around a cycle in a graph always satisfies the condition 
(l) of Kirchhoff’s law. Moreover, Kirchhoff’s equations are “homogeneous,” 
so the sum or difference of solutions to (l) yields another solution. Therefore 
we may conclude that the values of E 0 , E 2 , E 5 , . . . , E 25 are independent in the 
following sense: 

Ifxo, x 2 , ■ , . , x 2 5 are any real numbers (one Xj for each ej not in the free 
subtree G'), there is a solution to Kirchhoff’s equations (i) such that ( 4 ) 
E 0 = x 0 , E 2 = x 2 , . . . , E 2 5 = X 25 . 

Such a solution is found by going x 0 times around the cycle Co, x 2 times around 
cycle C 2 , etc. Furthermore, we find that the values of the remaining variables 
Ei, £ 3 , E 4 , . . . are completely dependent on the values E 0 , E 2 , ... . E 25 : 

The solution mentioned in statement ( 4 ) is unique. ( 5 ) 

For if there are two solutions to Kirchhoff’s equations such that Eo = Xo, ... , 
E 25 = x 2 $, we can subtract one from the other and we thereby obtain a solution 
in which Eo — E 2 = E 5 = • ■ • = E 23 = 0. But now all Ej must be zero, for it is 
easy to see that a nonzero solution to Kirchhoff’s equations is impossible when 
the graph is a free tree (see exercise 4). Therefore the two assumed solutions 
must be identical. We have now proved that all solutions of Kirchhoff’s equations 
may be obtained as sums of multiples of the fundamental cycles. 

When these remarks are applied to the graph in Fig. 32, we obtain the 
following general solution of Kirchhoff’s equations in terms of the independent 
variables Eq , E 2 , . . . , E 2 5 : 


Ex 

= Eq. 





Eu - 

= Eq, 




e 3 

- E 0 

- 

e 2 

+ 

E&, 

Ei 5 = 

= E i7 

- 

E 2 i, 


e 4 

= E 0 

- 

e 2 

+ 

Es, 

Eie = 

= En 

- 

e 2 1, 


e 6 

= E 0 

- 

e 5 

+ 

Es, 

El 8 - 

e'{q 

+ 

E 2 0 - 

- E ■ 

e 7 

— Eq 

- 

£5 

+ 

Es, 

E'i 9 = 

= eIq. 




Eq 

= Eq. 





E 22 - 

- En 

+ 

E 20 - 

- E- 

Ei 0 

= Eq. 





E 2 3 - 

= E 2 q. 




Eh 

= Eq 

+ 

Ei 7 - 

- E 2 i, 

E 2 4 - 

II 

<1 

- 

E 2 1, 


-Ei 2 

= Eq 

+ 

E^s, 


E 2 6 - 

= e 25 . 




E’i 3 

= K ' 3 , 




to 

-4 

= En 

- 

E 2 i - 

- E- 


To obtain these equations, we merely list, for each edge ej in the subtree, all Ek 
for which ej appears in cycle C*,, with the appropriate sign. [Thus, the matrix 
of coefficients in ( 6 ) is just the transpose of the matrix of coefficients in ( 3 ).] 
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Strictly speaking, Co should not be called a fundamental cycle, since it 
involves the special edge e 0 . We may call C 0 minus the edge e 0 a fundamental 
path from Start to Stop. Our boundary condition, that the Start and Stop boxes 
in the flow chart are performed exactly once, is equivalent to the relation 

E ° = L (7) 

The preceding discussion shows how to obtain all solutions to Kirchhoff’s 
law; the same method may be applied (as Kirchhoff himself applied it) to 
electrical circuits instead of program flow charts. It is natural to ask at this 
Point whether Kirchhoff’s law is the strongest possible set of equations that can 
be given for the case of program flow charts, or whether more can be said: Any 
execution of a computer program that goes from Start to Stop gives us a set of 
values Ei,E 2 ,..., E 27 for the number of times each edge is traversed, and these 
values obey Kirchhoff’s law; but are there solutions to Kirchhoff’s equations that 
do not correspond to any computer program execution? (I 11 this question, we do 
not assume that we know anything about the given computer program, except 
its flow chart.) If there are solutions that meet Kirclihoff’s conditions but do 
not correspond to actual program execution, we can give stronger conditions 
than Kirchhoff s law. For the case of electrical circuits Kirchhoff himself gave 
a second law [Ami. Physik und Chemie 64 (1845), 497-514]: The sum of the 
voltage drops around a fundamental cycle must be zero. This second law does 
not apply to our problem. 

There is indeed an obvious further condition that the E’s must satisfy, if 
they are to correspond to some actual path in the flow chart from Start to Stop: 
they must be integers, and in fact they must be nonnegative integers. This is 
not a trivial condition, since we cannot simply assign any arbitrary nonnegative 
integei values to the independent variables E 2 , E $ ..... E 27) ; for example, if we 
take E 2 = 2 and E 8 = 0, we find from ( 6 ) and ( 7 ) that E 3 = - 1 . (Thus, 
no execution of the flow chart in Fig. 31 will take branch e 2 twice without 
taking branch e 8 at least once.) The condition that all the E' s be nonnegative 
integers is not enough either; for example, consider the solution in which E[' g = 1 . 

E 5 = • • • = En = E 2q = E 2 i = E 2 5 — 0; there is no way to get to eig 
except via e 15 . The following condition is a necessary and sufficient condition 
that answers the problem raised in the previous paragraph: Let E 2 ,E 5 , ... . E 25 
be any given values, and determine E x , E z , . . . , E 27 according to (6), ( 7 ). Assume 
that all the E's are nonnegative integers, and assume that the graph whose edges 
are those ej for which E } > 0, and whose vertices are those that touch such ej. is 
connected. Then there is a path from Start to Stop in which edge e :j is traversed 
exactly Ej times. This fact is proved in the next section (see exercise 2.3.4.2-24). 

Let us now summarize the preceding discussion: 

Theorem K. If a How chart (such as Fig. 31) contains n boxes ( including Start 
and Stop) and m arrows, it is possible to find m — n + 1 fundamental cycles 
and a fundamental path from Start to Stop, such that any path from Start 
to Stop is equivalent (in terms of the number of times each edge is traversed) 
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to one traversal of the fundamental path plus a uniquely determined number 
of traversals of each of the fundamental cycles. (The fundamental path and 
fundamental cycles may include some edges that are to be traversed in a direction 
opposite that shown by the arrow on the edge; we conventionally say that such 
edges are being traversed —1 times.) 

Conversely, for any traversal of the fundamental path and the fundamental 
cycles in which the total number of times each edge is traversed is nonnegative, 
and in which the vertices and edges corresponding to a positive number of 
traversals form a connected graph, there is at least one equivalent path from 
Start to Stop. | 

The fundamental cycles are found by picking a free subtree as in Fig. 32; if 
we choose a different subtree we get, in general, a different set of fundamental 
cycles. The fact that there are m — n + 1 fundamental cycles follows from 
Theorem A. The modifications we made to get from Fig. 31 to Fig. 32, after 
adding e 0 , do not change the value of m — n + 1, although they may increase 
both m and n; the construction could have been generalized so as to avoid these 
trivial modifications entirely (see exercise 9). 

Theorem K is encouraging because it says that Kirchhoff’s law (which con- 
sists of n equations in the m unknowns E\, E 2 , . ■ . , E m ) has just one “redun- 
dancy” : These n equations allow us to eliminate n — 1 unknowns. However, the 
unknown variables throughout this discussion have been the number of times the 
edges have been traversed, not the number of times each box of the flow chart 
has been entered. Exercise 8 shows how to construct another graph whose edges 
correspond to the boxes of the flow chart, so that the theory above can be used 
to deduce the true number of redundancies between the variables of interest. 

Applications of Theorem K to software for measuring the performance of 
programs in high-level languages are discussed by Thomas Ball and James R. 
Larus in ACM Trans. Prog. Languages and Systems 16 (1994), 1319-1360. 

EXERCISES 

1. [14] List all cycles from B to B that are present in the graph of Fig. 29. 

2 . [M20] Prove that if V and V are vertices of a graph and if there is a path from 
V to V\ then there is a simple path from V to V'. 

3. [15] What path from Start to Stop is equivalent (in the sense of Theorem K) to 
one traversal of the fundamental path plus one traversal of cycle C 2 in Fig. 32? 

► 4. [M20] Let G' be a finite free tree in which arrows have been drawn on its edges 
ei,...,e n -i; let E \, . . . , f?„_i be numbers satisfying Kirchhoff’s law ( 1 ) in G'. Show 
that Ei = • • • = E n - 1 = 0. 

5. [20] Using Eqs. ( 6 ), express the quantities A, B, . . . , S that appear inside the 
boxes of Fig. 31 in terms of the independent variables E 2 ,E$, . . . , U 25 . 

► 6 . [M2 7] Suppose a graph has n vertices Vi, ... ,V n and m edges ei, . . , ,e m . Each 
edge e is represented by a pair of integers (a, b) if it joins V a to Vb. Design an algorithm 
that takes the input pairs (ai, 61 ), . . . , (a m , b m ) and prints out a subset of edges that 
forms a free tree; the algorithm reports failure if this is impossible. Strive for an efficient 
algorithm. 
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7. [ 22 ] Carry out the construction in the text for the flow chart 



using the free subtree consisting of edges ei, e 2 , e 3 , e 4 , eg. What are the fundamental 
cycles? Express E u E 2 , E 3 , E 4 , Eg in terms of E s , E s , E 7 , and E a . 

► 8. [M25] When applying Kirchhoff’s first law to program flow charts, we usually are 
interested only in the vertex flows (the number of times each box of the flow chart 
is performed), not the edge flows analyzed in the text. For example, in the graph of 
exercise 7, the vertex flows are A = E 2 + E 4 , B = E 5 , C = E 3 + E 7 + Eg, D = E 6 + Eg. 

If we group some vertices together, treating them as one “supervertex,” we can 
combine edge flows that correspond to the same vertex flow. For example, edges e 2 
and e 4 can be combined in the flow chart above if we also put B with D: 



(Here e 0 has also been added from Stop to Start, as in the text.) Continuing this 
procedure, we can combine e 3 + e 7 , then (e 3 + e 7 ) + e 8 , then e 6 + e 9 , until we obtain 
the reduced flow chart having edges s = ei , a = e 2 + e 4 , b = e 5 , c = e 3 + df + e 8 , 
d = e 6 + e 9 , t = e 0 , precisely one edge for each vertex in the original flow chart: 



By construction, Kirchhoff s law holds in this reduced flow chart. The new edge 
flows are the vertex flows of the original; hence the analysis in the text, applied to the 
reduced flow chart, shows how the original vertex flows depend on each other. 

Prove that this reduction process can be reversed, in the sense that any set of 
flows {a, b, . . .} satisfying Kirchhoff’s law in the reduced flow chart can be “split up” 
into a set of edge flows {e 0 ,ei, . . .} in the original flow chart. These flows ej satisfy 
Kirchhoff’s law and combine to yield the given flows {a,b, . . .}; some of them might, 
however, be negative. (Although the reduction procedure has been illustrated here for 
only one particular flow chart, your proof should be valid in general.) 

9. [M22] Edges ei 3 and ei 9 were split into two parts in Fig. 32, since a graph is not 
supposed to have two edges joining the same two vertices. However, if we look at the 
final result of the construction, this splitting into two parts seems quite artificial since 
E 13 = E " 3 and E[ 9 = E" g are two of the relations found in (6), while E" 3 and E" g are 
two of the independent variables. Explain how the construction could be generalized 
so that an artificial splitting of edges may be avoided. 
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10. [16] An electrical engineer, designing the circuitry for a computer, has n terminals 
T \ , TV , . . . , T n that should be at essentially the same voltage at all times. To achieve 
this, the engineer can solder wires between any pairs of terminals; the idea is to make 
enough wire connections so that there is a path through the wires from any terminal to 
any other. Show that the minimum number of wires needed to connect all the terminals 
is n — 1, and n — 1 wires achieve the desired connection if and only if they form a free 
tree (with terminals and wires standing for vertices and edges). 

11. [M27] (R. C. Prim, Bell System Tech. J. 36 (1957), 1389-1401.) Consider the 
wire connection problem of exercise 10 with the additional proviso that a cost c(i,j) is 
given for each i < j , denoting the expense of wiring terminal T to terminal Tj. Show 
that the following algorithm gives a connection tree of minimum cost: “If n = 1, do 
nothing. Otherwise, renumber terminals { 1 , . , , , n— 1} and the associated costs so that 
c(n - 1 , n) = mini<j <n c(i, n); connect terminal T n _ 1 to T n ; then change c(j , n - 1) 
to min(c(j, n — 1), c(j,n)) for 1 < j < n — 1, and repeat the algorithm for n — 1 
terminals 7 1 ... . ,T n ~ 1 using these new costs. (The algorithm is to be repeated with 
the understanding that whenever a connection is subsequently requested between the 
terminals now called Tj and T n - 1, the connection is actually made between terminals 
now called Tj and T n if it is cheaper; thus T n _ 1 and T„ are being regarded as though 
they were one terminal in the remainder of the algorithm.)” This algorithm may also 
be stated as follows: “Choose a particular terminal to start with; then repeatedly make 
the cheapest possible connection from an unchosen terminal to a chosen one, until all 
have been chosen.” 


Fig. 33. Free tree 
of minimum cost. 
(See exercise 11.) 
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For example, consider Fig. 33(a), which shows nine terminals on a grid; let the 
cost of connecting two terminals be the wire length, namely the distance between them. 
(The reader may wish to try to And a minimal cost tree by hand, using intuition instead 
of the suggested algorithm.) The algorithm would first connect Ts to Tg, then Tg to 
Tg, Tg to Tg, T2 to Tg, T\ to T2, T3 to Ti, T7 to T3, and finally T4 to either T2 or Tg. 
A minimum cost tree (wire length 7 + 2 \fl + 2 \/5 ) is shown in Fig. 33(b). 

► 12. [29] The algorithm of exercise 11 is not stated in a fashion suitable for direct 
computer implementation. Reformulate that algorithm, specifying in more detail the 
operations that are to be done, in such a way that a computer program can carry out 
the process with reasonable efficiency. 

13. [M2 If] Consider a graph with n vertices and m edges, in the notation of exercise 6. 
Show that it is possible to write any permutation of the integers {1,2, ...,n} as 
a product of transpositions (dkfbkf) (a,k 2 bk 2 ) ■ ■ ■ (ak t bk t ) if and only if the graph is 
connected. (Hence there are sets of n — 1 transpositions that generate all permutations 
on n elements, but no set of n — 2 will do so.) 
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2. 3. 4. 2. Oriented trees. In the previous section, we saw that an abstracted 
flow chart may be regarded as a graph, if we ignore the direction of the arrows 
on its edges; the graph-theoretic ideas of cycle, free subtree, etc., were shown to 
be relevant in the study of flow charts. There is a good deal more that can be 
said when the direction of each edge is given more significance, and in this case 
we have what is called a “directed graph” or “digraph.” 

Let us define a directed graph formally as a set of vertices and a set of arcs , 
each arc leading from a vertex V to a vertex V'. If e is an arc from V to V' 
we say V is the initial vertex of e, and V' is the final vertex, and we write 

V = init(e), V' = fin(e). The case that init(e) = fin(e) is not excluded (although 
it was excluded from the definition of edge in an ordinary graph), and several 
different arcs may have the same initial and final vertices. The out-degree of a 
vertex V is the number of arcs leading out from it, namely the number of arcs e 
such that init(e) = V ; similarly, the in-degree of V is defined to be the number 
of arcs with fin(e) = V. 

The concepts of paths and cycles are defined for directed graphs in a manner 
similar to the corresponding definitions for ordinary graphs, but some important 
new technicalities must be considered. If e 1 ,e 2 ,. . . ,e n are arcs (with n > T), 
we say that (ei, e 2 , . . . , e n ) is an oriented path of length n from V to V' if 

V — init(ei), V = fin(e n ), and fin(efc) = init(efc + i) for 1 < k < n. An oriented 
path (ei,e 2 ,...,e n ) is called simple if init(ei), ..., init(e n ) are distinct and 
fin(ei), ..., fin(e n ) are distinct. An oriented cycle is a simple oriented path 
from a vertex to itself. (An oriented cycle can have length 1 or 2, but such short 
cycles were excluded from our definition of “cycle” in the previous section. Can 
the reader see why this makes sense?) 

As examples of these straightforward definitions, we may refer to Fig. 31 in 
the previous section. The box labeled “J” is a vertex with in-degree 2 (because 
of the arcs ei 6 , e 2 i) and out-degree 1. The sequence (en, e^eis, e 22 ) is an 
oriented path of length 4 from J to P\ this path is not simple since, for example, 
init(eig) = L = init(e 22 ). The diagram contains no oriented cycles of length 1. 
but (e i8 , ei 9 ) is an oriented cycle of length 2. 

A directed graph is said to be strongly connected if there is an oriented path 
from V to V for any two vertices V ± V'. It is said to be rooted if there is at 
least one root , that is, at least one vertex R such that there is an oriented path 
from V to R for all Vfi R. “Strongly connected” always implies “rooted,” but 
the converse does not hold. A flow chart such as Fig. 31 in the previous section 
is an example of a rooted digraph, with R the Stop vertex; with the additional 
arc from Stop to Start (Fig. 32) it becomes strongly connected. 

Every directed graph G corresponds in an obvious manner to an ordinary 
graph Go, if we ignore orientations and discard duplicate edges or loops. For- 
mally speaking, G 0 has an edge from V to V' if and only iiVfiV' and G has an 
arc from V to V or from V' to V. We can speak of (unoriented) paths and cycles 
in G with the understanding that these are paths and cycles of G 0 ; we can say 
that G is connected — this is a much weaker property than “strongly connected,” 
even weaker than “rooted” — if the corresponding graph G 0 is connected. 


2. 3.4. 2 

An oriented tree (see Fig. 34), sometimes called 
a "rooted tree” by other authors, is a directed graph 
with a specified vertex R such that: 

a) Each vertex V / R is the initial vertex of exactly 
one arc, denoted by e[V], 

b) R is the initial vertex of no arc; 

c) R is a root in the sense defined above (that is, 
for each vertex V yf R there is an oriented path 
from V to R). 

It follows immediately that for each vertex V^R 
there is a unique oriented path from V to R ; and 
hence there are no oriented cycles. 

Our previous definition of “oriented tree” (at the beginning of Section 2.3) 
is easily seen to be compatible with the new definition just given, when there 
are finitely many vertices. The vertices correspond to nodes, and the arc e[V] is 
the link from V to PARENT [ V] . 

The (undirected) graph corresponding to an oriented tree is connected, 
because of property (c). Furthermore, it has no cycles. For if (Vo, Vi, . . . , V n ) 
is an undirected cycle with n > 3, and if the edge between V 0 and Vf is e[Vi], 
then the edge between Vi and V 2 must be e[V 2 \, and similarly the edge between 
Vk~i and Vi must be e[Vj,] for 1 < k < n, contradicting the absence of oriented 
cycles. If the edge between V 0 and Vi is not e[Vi], it must be e[Vb], and the same 
argument applies to the cycle 
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R 



(Vi^cK-i,...,^), 

because V n = V Q . Therefore an oriented tree is a free tree when the direction of 
the arcs is neglected. 

Conversely, it is important to note that we can reverse the process just 
described. If we start with any nonempty free tree, such as that in Fig. 30, we 
can choose any vertex as the root R, and assign directions to the edges. The 
intuitive idea is to “pick up” the graph at vertex R and shake it; then assign 
upward-pointing arrows. More formally, the rule is this: 

Change the edge V — V' to an arc from V to V' if and only if the simple 
path from V to R leads through V', that is, if it has the form (Vo, Vi, . . . , V n ), 
where n > 0, V 0 = V, Vi = V\ V n = R. 


To verify that such a construction is valid, we need to prove that each edge 
V — V is assigned the direction Vi — V' or the direction V — >V'\ and this 
is easy to prove, for if (V) Vf, . . . ,R) and (V\ V{, . . . , R) are simple paths, there 
is a cycle unless V = V{ or Vi = V'. This construction demonstrates that the 
directions of the arcs in an oriented tree are completely determined if we know 
which vertex is the root, so they need not be shown in diagrams when the root 
is explicitly indicated. 
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Fig. 35. Three tree structures. 


We now see the relation between three types of trees: the (ordered) tree, 
which is of principal importance in computer programs, as defined at the be- 
ginning of Section 2.3; the oriented tree (or unordered tree); and the free tree. 
Both of the latter two types arise in the study of computer algorithms, but not 
as often as the first type. The essential distinction between these types of tree 
structure is merely the amount of information that is taken to be relevant. For 
example, Fig. 35 shows three trees that are distinct if they are considered as 
ordered trees (with root at the top). As oriented trees, the first and second are 
identical, since the left-to-right order of subtrees is immaterial; as free trees, all 
three graphs in Fig. 35 are identical, since the root is immaterial. 

An Eulerian trail in a directed graph is an oriented path (e x , e 2 , . . . , e m ) such 
that every arc in the directed graph occurs exactly once, and fin(e m ) = init( ei ). 
This is a “complete traversal” of the arcs of the digraph. (Eulerian trails get 
their name from Leonhard Euler’s famous discussion in 1736 of the impossibility 
of traversing each of the seven bridges in the city of Konigsberg exactly once 
during a Sunday stroll. He treated the analogous problem for undirected graphs. 
Eulerian trails should be distinguished from “Hamiltonian cycles,” which are 
oriented cycles that encounter each vertex exactly once; see Chapter 7.) 

A directed graph is said to be balanced (see Fig. 36) if every vertex V has 
the same in-degree as its out-degree, that is, if there are just as many edges 
with V as their initial vertex as there are with V as their final vertex. This 
condition is closely related to Kirchhoff’s law (see exercise 24). If a directed 
graph has an Eulerian trail, it must obviously be connected and balanced — 
unless it has isolated vertices, which are vertices with in-degree and out-degree 
both equal to zero. 

So far m this section we’ve looked at quite a few definitions (directed graph, 
arc, initial vertex, final vertex, out-degree, in-degree, oriented path, simple 
oriented path, oriented cycle, oriented tree, Eulerian trail, isolated vertex, and 
the properties of being strongly connected, rooted, and balanced), but there 
has been a scarcity of important results connecting these concepts. Now we are 
ready for meatier material. The first basic result is a theorem due to I. J. Good 
[J. London Math. Soc. 21 (1947), 167-169], who showed that Eulerian trails are 
always possible unless they are obviously impossible: 
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Fig. 36. A balanced directed graph. 


Theorem G. A finite, directed graph with no isolated vertices possesses an 
Eulerian trail if and only if it is connected and balanced. 

Proof. Assume that G is balanced, and let 

P (ei , . . . , e m ) 

be an oriented path of longest possible length that uses no arc twice. Then if 
V = fin(e m ), and if k is the out-degree of V, all k arcs e with init(e) = V must 
already appear in P\ otherwise we could add e and get a longer path. But if 
init(ej) = V and j > 1, then fin^j-i) = V. Hence, since G is balanced, we must 
have 

init(ei) = V — fin(e m ), 

otherwise the in-degree of V would be at least k + 1. 

Now by the cyclic permutation of P it follows that any arc e not in the path 
has neither initial nor final vertex in common with any arc in the path. So if 
P is not an Eulerian trail, G is not connected. | 

There is an important connection between Eulerian trails and oriented trees: 

Lemma E. Let (ei, . . . , e m ) be an Eulerian trail of a directed graph G having 
no isolated vertices. Let R = fin(e m ) = init(ei). For each vertex V ^ R let e[V] 
be the last exit from V in the trail; that is, 

e[V] — e } if init(ej) = V and init(e*,) / V for j < k < m. (i) 

Then the vertices of G with the arcs e[V] form an oriented tree with root R. 

Proof. Properties (a) and (b) of the definition of oriented tree are evidently 
satisfied. By exercise 7 we need only show that there are no oriented cycles 
among the e[V]; but this is immediate, since if fin(e[V]) — V' — init(e[H']), 
where e[V] = e } and e[V'] = ey, then j < j'. | 

This lemma can perhaps be better understood if we turn things around and 
consider the “first entrances” to each vertex; the first entrances form an un- 
ordered tree with all arcs pointing away from R. Lemma E has a surprising and 
important converse, proved by T. van Aardenne-Ehrenfest and N. G. de Bruijn 
[Simon Stevin 28 (1951), 203-217]: 
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Theorem D. Let G be a finite, balanced , directed graph, and let G’ be an 
oriented tree consisting of the vertices of G plus some of the arcs of G. Let R 
be the root of G' and let e[V] be the arc of G' with initial vertex V. Let e t be 
any arc of G with init( ei ) = R. Then P = (e u e 2 , . . . , e m ) is an Eulerian trail if 
it is an oriented path for which 

i) no arc is used more than once ; that is, ej y? e k when j k. 

ii) e[V] is not used in P unless it is the only choice consistent with rule (i); 
that is, if ej = e[V] and if e is an arc with init(e) = V, then e = e k for some 

k<j. 

iii) P terminates only when it cannot be continued by rule (i); that is, if 
init(e) = fin(e m ), then e = e k for some k. 

Proof. By (iii) and the argument in the proof of Theorem G, we must have 
fin(e m ) = init(ei) = R. Now if e is an arc not appearing in P, let V = fin(e). 
Since G is balanced, it follows that V is the initial vertex of some arc not in P: 
and if V / R, condition (ii) tells us that e[V] is not in P. Now use the same 
argument with e = e[V], and we ultimately find that R is the initial vertex of 
some arc not in the path, contradicting (iii). | 

The essence of Theorem D is that it shows us a simple way to construct an 
Eulerian trail in a balanced directed graph, given any oriented subtree of the 
graph. (See the example in exercise 14.) In fact, Theorem D allows us to count 
the exact number of Eulerian trails in a directed graph; this result and many 
other important consequences of the ideas developed in this section appear in 
the exercises that follow. 

EXERCISES 

1. [M20] Prove that if V and V' are vertices of a directed graph and if there is an 
oriented path from V to V, then there is a simple oriented path from V to V. 

2. [15] Which of the ten “fundamental cycles” listed in (3) of Section 2.3.4. 1 are 
oriented cycles in the directed graph (Fig. 32) of that section? 

3. [16] Draw the diagram for a directed graph that is connected but not rooted. 

4. [M20] The concept of topological sorting can be defined for any finite directed 
graph G as a linear arrangement of the vertices V 1 V 2 ...V n such that init(e) precedes 
fin(e) in the ordering for all arcs e of G. (See Section 2.2.3, Figs. 6 and 7.) Not all finite 
directed graphs can be topologically sorted; which ones can be? (Use the terminology 
of this section to give the answer.) 

5. [ M16 ] Let G be a directed graph that contains an oriented path (ei, . . . ,e„) with 
fin(e„) = mit(e 1 ). Give a proof that G is not an oriented tree, using the terminology 
defined in this section. 

6. [M21] True or false: A directed graph that is rooted and contains no cycles and 
no oriented cycles is an oriented tree. 

7. [M22] True or false: A directed graph satisfying properties (a) and (b) of the 
definition of oriented tree, and having no oriented cycles, is an oriented tree. 
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8. Study the properties of automorphism groups of oriented trees, namely 
the groups consisting of all permutations n of the vertices and arcs for which we have 
init(e7r) = init(e)7r, fin(e7r) = fin(e)7r. 

9. [18] By assigning directions to the edges, draw the oriented tree corresponding to 
the free tree in Fig. 30 on page 363, with G as the root. 

10. [22] An oriented tree with vertices 14 , . . . , V„ can be represented inside a computer 
by using a table P[l], . . . , P[n] as follows: If Vj is the root, P[j] = 0; otherwise P[j] — k, 
if the arc e[V 3 ] goes from Vj to 14. (Thus P[l], . . . , P[n] is the same as the “parent” 
table used in Algorithm 2.3.3E.) 

The text shows how a free tree can be converted into an oriented tree by choosing 
any desired vertex to be the root. Consequently, it is possible to start with an 
oriented tree that has root P, then to convert this into a free tree by neglecting the 
orientation of the arcs, and finally to assign new orientations, obtaining an oriented 
tree with any specified vertex as the root. Design an algorithm that performs this 
transformation: Starting with a table P[l], . . . , P[n], representing an oriented tree, and 
given an integer j, 1 < j < n, design the algorithm to transform the P table so that it 
represents the same free tree but with Vj as the root. 

► 11. [28] Using the assumptions of exercise 2.3.4. 1-6, but with ( a k ,bk ) representing 
an arc from V ak to \% k , design an algorithm that not only prints out a free subtree as 
in that algorithm, but also prints out the fundamental cycles. [Hint: The algorithm 
given in the solution to exercise 2.3.4. 1-6 can be combined with the algorithm in the 
preceding exercise.] 

12. [M10] In the correspondence between oriented trees as defined here and oriented 
trees as defined at the beginning of Section 2.3, is the degree of a tree node equal to 
the in-degree or the out-degree of the corresponding vertex? 

► 13. [M24] Prove that if R is a root of a (possibly infinite) directed graph G, then 
G contains an oriented subtree with the same vertices as G and with root R. (As a 
consequence, it is always possible to choose the free subtree in flow charts like Fig. 32 
of Section 2.3.4. 1 so that it is actually an oriented subtree; this would be the case in 
that diagram if we had selected e" 3 , e" g , e 20 , and en instead of e' 13 , e' 19 , e 2 3 , and ei 5 .) 

14. [21] Let G be the balanced digraph shown in Fig. 36, and let G' be the oriented 
subtree with vertices Vo, 14, U 2 and arcs eoi, e 2 i. Find all paths P that meet the 
conditions of Theorem D, starting with arc ei 2 . 

15. [M20] True or false: A directed graph that is connected and balanced is strongly 
connected. 

► 16. [M24] In a popular solitaire game called “clock,” the 52 cards of an ordinary deck 
of playing cards are dealt face down into 13 piles of four each; 12 piles are arranged 
in a circle like the 12 hours of a clock and the thirteenth pile goes in the center. The 
solitaire game now proceeds by turning up the top card of the center pile, and then 
if its face value is k, by placing it next to the fcth pile. (The numbers 1, 2, . . . , 13 are 
equivalent to A, 2, ... , 10, J, Q, K.) Play continues by turning up the top card of the 
fcth pile and putting it next to its pile, etc., until we reach a point where we cannot 
continue since there are no more cards to turn up on the designated pile. (The player 
has no choice in the game, since the rules completely specify what to do.) The game is 
won if all cards are face up when play terminates. [Reference: E. D. Cheney, Patience 
(Boston: Lee & Shepard, 1870), 62-65; the game was called “Travellers’ Patience” in 
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England, according to M. Whitmore Jones, Games of Patience (London: L Upcott 
Gill, 1900), Chapter 7.] 

Show that the game will be won if and only if the following directed graph is an 
oriented tree: The vertices are Vi, V 2 , . . . , V'i 3 ; the arcs are ei, e 2 , . . . , ei 2 , where e, goes 
from Vj to Vk if k is the bottom card in pile j after the deal. 

(In particular, if the bottom card of pile j is a " j‘‘ . for j ^ 13, it is easy to see 
that the game is certainly lost, since this card could never be turned up. The result 
proved in this exercise gives a much faster way to play the game!) 

17 . [M32] What is the probability of winning the solitaire game of clock (described 
in exercise 16), assuming the deck is randomly shuffled? What is the probability that 
exactly k cards are still face down when the game is over? 

18 . [ M30 ] Let G be a graph with n + 1 vertices V 0 ,V u ...,V n and m edges ei, . . . , e m . 
Make G into a directed graph by assigning an arbitrary orientation to each edge; then 
construct the m x (n + 1) matrix A with 

( +1, if init(ej) = Vj ; 
a ij = < -1, if fin(ei) = Vj ; 

( 0 , otherwise. 

Let A 0 be the m x n matrix A with column 0 deleted. 

a) If m = n, show that the determinant of A 0 is equal to 0 if G is not a free tree, and 
equal to ±1 if G is a free tree. 

b) Show that for general m the determinant of A^Aq is the number of free subtrees 
of G (namely the number of ways to choose n of the to edges so that the resulting 
graph is a free tree). [Hint: Use (a) and the result of exercise 1.2.3-46.] 

19 . [MSI] ( The matrix tree theorem.) Let G be a directed graph with n + 1 vertices 

Vo, Vi, . . . , V n . Let A be the (n + 1) x (n + 1) matrix with 

_f — k, if i / j and there are k arcs from Vi to V); 

13 \ t, if * = j and there are 1, arcs from Vj to other vertices. 

(It follows that dio + an T * * * T a. in = 0 for 0 < i < n.) Let Ao be the same matrix 

with row 0 and column 0 deleted. For example, if G is the directed graph of Fig. 36, 
we have 



a) Show that if aoo — 0 and ajj — 1 for 1 < j < n, and if G contains no arcs from a 
vertex to itself, then (let A 0 = [G is an oriented tree with root Vo]. 

b) Show that in general, det .4o is the number of oriented subtrees of G rooted at Vo 
(namely the number of ways to select n of the arcs of G so that the resulting 
directed graph is an oriented tree, with Vo as the root). [Hint: Use induction on 
the number of arcs.] 

20. [M21 ] If G is an undirected graph on n + 1 vertices Vo, ... , V n , let B be the nxn 
matrix defined as follows for 1 < i,j < n: 

{ t, if i = j and there are t edges touching V); 

— 1, if i 7 ^ j and V, is adjacent to V); 

0 , otherwise. 
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For example, if G is the graph of Fig. 29 on page 363, with (V 0 , V, V 2 , V 3 , V 4 ) = 
( A, B,C, D, E ), we find that 


/ 3 0-1 -1\ 



\-l 0 -1 2/ 


Show that the number of free subtrees of G is det B. [Hint: Use exercise 18 or 19.] 

21. [HM38] (T. van Aardenne-Ehrenfest and N. G. de Bruijn.) Figure 36 is an 
example of a directed graph that is not only balanced, it is regular, which means that 
every vertex has the same in-degree and out-degree as every other vertex. Let G be 
a regular digraph with n vertices Vo, Vi, ... , V n _i, in which every vertex has in-degree 
and out-degree equal to to. (Hence there are mn arcs in all.) Let G* be the digraph 
with mn vertices corresponding to the arcs of G; let a vertex of G* corresponding to 
an arc from V) to \\- in G be denoted by V}*,. An arc goes from Vjk to Vy^' in G* if 
and only if k = j'. For example, if G is the directed graph of Fig. 36, G* is shown in 
Fig. 37. An Eulerian trail in G is a Hamiltonian cycle in G* and conversely. 

Prove that the number of oriented subtrees of G* is TO (m_1)n times the number of 
oriented subtrees of G. [Hint: Use exercise 19.] 



380 INFORMATION STRUCTURES 


2 . 3 . 4. 2 


► 22. [M26] Let G be a balanced, directed graph with vertices Vi,V 2 ,...,V n and no 
isolated vertices. Let <r, be the out-degree of Vj . Show that the number of Eulerian 
trails of G is 

n 

{(7 1 +a 2 + \-(r„)T - 1)!, 

l=i 

where T is the number of oriented subtrees of G with root V). [Note: The factor 
(<ri + • • • + <r„), which is the number of arcs of G, may be omitted if the Eulerian trail 
(ei,...,e m ) is regarded as equal to (e k , . . . , e m , ei, . . . , e fc _i).] 

► 23. [M3 3] (N. G. de Bruijn.) For each sequence of nonnegative integers xi , . . . , x k 
less than m, let f(x k , . . . , x k ) be a nonnegative integer less than m. Define an infinite 
sequence as follows: X k = X 2 = --- = X k = 0; X n+k+1 = f(X n+k ,...,X n+1 ) when 
n > 0. For how many of the m m possible functions / is this sequence periodic 
with a period of the maximum length m k ? [Hint: Construct a directed graph with 
vertices (xi, . . . , xs,-i) for all 0 < xj < m, and with arcs from (*i, x 2 , . . . , x k -i) to 
(x 2 , . . . , x k -i, x k )\ apply exercises 21 and 22.] 

►24. [M20] Let G be a connected digraph with arcs e 0 ,ei,...,e m . Let E 0 , E lt . . . , E m 

be a set of positive integers that satisfy Kirchhoff’s law for G; that is, for each vertex V, 

E E 

imt(ej)=V fin(e 3 ) = V 

Assume further that E 0 = 1. Prove that there is an oriented path in G from fin(e 0 ) to 
mit(eo) such that edge e 3 appears exactly Ej times, for 1 < j < m, while edge e 0 does 
not appear. [Hint: Apply Theorem G to a suitable directed graph.] 

25. [26] Design a computer representation for directed graphs that generalizes the 
right-threaded binary tree representation of a tree. Use two link fields ALINK, BLINK 
and two one-bit fields ATAG, BTAG; and design the representation so that: (i) there is 
one node for each arc of the directed graph ( not for each vertex); (ii) if the directed 
graph is an oriented tree with root R, and if we add an arc from R t o a new vertex H. 
then the representation of this directed graph is essentially the same as a right-threaded 
representation of this oriented tree (with some order imposed on the children in each 
family), in the sense that ALINK, BLINK, BTAG are respectively the same as LLINK, 
ELINK, RTAG in Section 2.3.2; and (iii) the representation is symmetric in the sense that 
interchanging ALINK, ATAG, with BLINK, BTAG is equivalent to changing the direction on 
all the arcs of the directed graph. 

► 26. [HM39] ( Analysis of a random algorithm .) Let G be a directed graph on the 
vertices Vi, V 2 , . . . , V n . Assume that G represents the flow chart for an algorithm, 
where Vi is the Start vertex and V n is the Stop vertex. (Therefore V n is a root of G.) 
Suppose each arc e of G has been assigned a probability p(e), where the probabilities 
satisfy the conditions 

0 < p(e) <1; ^ P(e) = 1 for 1 < j < n. 

init (e) = Vj 

Consider a random path, which starts at Vi and subsequently chooses branch e of G 
with probability p(e), until V n is reached; the choice of branch taken at each step is to 
be independent of all previous choices. 
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For example, consider the graph of exercise 2.3.4. 1-7, and assign the respective 
probabilities 1, |, 1, 1, |, 1, I, 1 to arcs ei,e 2 , . . . , e 9 . Then the path “Start- A- 

B-C-A-D-B-C- Stop” is chosen with probability l-|-l'|-|-f-l | = ~ • 

Such random paths are called Markov chains, after the Russian mathematician 
Andrei A. Markov, who first made extensive studies of stochastic processes of this 
kind. The situation serves as a model for certain algorithms, although our requirement 
that each choice must be independent of the others is a very strong assumption. The 
purpose of this exercise is to analyze the computation time for algorithms of this kind. 

The analysis is facilitated by considering the n x n matrix A = (aij), where 
a ij = El>( e ) summed over all arcs e that go from V, to V 3 . If there is no such arc, 
aij = 0. The matrix A for the example considered above is 

/0 1 0 0 0 0\ 

0 0 i 0 i 0 

0 0 0 1 0 0 

0-00-4 

u 2 u u 4 4 

0 o f 0 0 i 

Vo 0 0 0 0 0/ 

It follows easily that ( A k ) i3 is the probability that a path starting at Vi will be at Vj 
after k steps. 

Prove the following facts, for an arbitrary directed graph G of the stated type: 

a) The matrix (7 — A) is nonsingular. [Hint: Show that there is no nonzero vector x 
with xA n = x.[ 

b) The average number of times that vertex Vj appears in the path is 

(I - A)f/ = cofactor J1 (7 - A)/det(I - A), for 1 < j < n. 

[Thus in the example considered we find that the vertices A, B, C, D are traversed 
respectively — , |, |, | times, on the average.] 

c) The probability that Vj occurs in the path is 

a 3 = cofactor, i(7 - A)/cofactorj (7 - A); 

furthermore, a n = 1, so the path terminates in a finite number of steps with 
probability one. 

d) The probability that a random path starting at V, will never return to V, is 
bj = det (7 — A) /cofactor jj (7 — A). 

e) The probability that Vj occurs exactly k times in the path is aj(l - b J ) k ~ 1 b j , for 
k > 1, 1 < j < n. 

27. [ M30 ] ( Steady states.) Let G be a directed graph on vertices Vi, . . . , V n , whose 
arcs have been assigned probabilities p(e) as in exercise 26. Instead of having Start 
and Stop vertices, however, assume that G is strongly connected; thus each ver- 
tex Vj is a root, and we assume that the probabilities p(e) are positive and satisfy 
Sinit(e)=v 3 P ( e ) = 1 f° r a11 J- A random process of the kind described in exercise 26 is 
said to have a “steady state” (xi, . . . ,x n ) if 

Xj = ^ p(e) Xi, for 1 < j < n. 

init(e)=Vi 
fin (e) = Vj 
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Let tj be the sum, over all oriented subtrees T 3 of G that are rooted at Vj, of the 
products p{ e )- Prove that (ti, . . . , t n ) is a steady state of the random process. 

► 28. [M3 5} Consider the (m + n) x ( m + n ) determinant illustrated here for m = 2 and 
n = 3: 

^aio + an + a i2 + ai3 0 an a i2 m 3 \ 

0 a 2 o + a 2 i + a 22 + a 23 a 2 i a 22 a 2 3 

det 6n &12 bio + bn + b 12 0 0 

^21 b 22 0 b 2 o + b 2 1 + b 22 0 

^ 3 i h 32 0 0 6 3 o + 631 + 632 / 

Show that when this determinant is expanded as a polynomial in the a’s and 6’s, each 
nonzero term has coefficient +1. How many terms appear in the expansion? Give a 
rule, related to oriented trees, that characterizes exactly which terms are present. 

*2. 3. 4. 3. The “infinity lemma.” Until now we have concentrated mainly on 
trees that have only finitely many vertices (nodes), but the definitions we have 
given for free trees and oriented trees apply to infinite graphs as well. Infinite 
ordered trees can be defined in several ways; we can, for example, extend the 
concept of Dewey decimal notation to infinite collections of numbers, as in 
exercise 2.3-14. Even in the study of computer algorithms there is occasionally 
a need to know the properties of infinite trees — for example, to prove by contra- 
diction that a certain tree is not infinite. One of the most fundamental properties 
of infinite trees, first stated in its full generality by D. Konig, is the following: 

Theorem K. ( The “infinity lemma. ”) Every infinite oriented tree in which 
every vertex has finite degree has an infinite path to the root, that is, an infinite 
sequence of vertices V 0 , V 1 ,V 2 ,... in which V 0 is the root and fm(e[V) +1 ]) = U 
for all j > 0. 

Proof. We define the path by starting with Vo, the root of the oriented tree. 
Assume that j > 0 and that Vj has been chosen having infinitely many de- 
scendants. The degree of Vj is finite by hypothesis, so Vj has finitely many 
children Ui, ... , U n . At least one of these children must possess infinitely many 
descendants, so we take V J+1 to be such a child of Vj. 

Now V 0 , Vi, V 2 , ... is an infinite path to the root. | 

Students of calculus may recognize that the argument used here is essentially 
like that used to prove the classical Bolzano- Weierstrass theorem, “A bounded 
infinite set of real numbers has an accumulation point.” One way of stating 
Theorem K, as Konig observed, is this: “If the human race never dies out, 
somebody now living has a line of descendants that will never die out.” 

Most people think that Theorem K is completely obvious when they first 
encounter it, but after more thought and a consideration of further examples 
they realize that there is something profound about it. Although the degree of 
each node of the tree is finite, we have not assumed that the degrees are bounded 
(less than some number N for all vertices), so there may be nodes with higher 
and higher degrees. It is at least conceivable that everyone’s descendants will 
ultimately die out although there will be some families that go on a million 
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generations, others a billion, and so on. In fact, H. W. Watson once published a 
“proof that under certain laws of biological probability carried out indefinitely, 
there will be infinitely many people born in the future but each family line will 
die out with probability one. His paper [J. Anthropological Inst. Gt. Britain and 
Ireland 4 (1874), 138-144] contains important and far-reaching theorems in spite 
of the minor slip that caused him to make this statement, and it is significant 
that he did not find his conclusions to be logically inconsistent. 

The contrapositive of Theorem K is directly applicable to computer algo- 
rithms: If we have an algorithm that periodically divides itself up into finitely 
many subalgorithms, and if each chain of subalgorithms ultimately terminates, 
then the algorithm itself terminates. 

Phrased yet another way, suppose we have a set S, finite or infinite, such 
that each element of S' is a sequence (aq, aq, . ■ ■ , x n ) of positive integers of finite 
length n > 0. If we impose the conditions that 

i) if (aq, ...,x n ) is in S, so is (* 1 , . . . , x k ) for 0 < k < n; 

ii) if (xi,...,x n ) is in S, only finitely many x n+ i exist for which (aq,..., 
x n , x n+ i) is also in S; 

iii) there is no infinite sequence (aq,aq,...) all of whose initial subsequences 
(aq,aq , . . ,,i„) lie in S; 

then S is essentially an oriented tree, specified essentially in a Dewey decimal 
notation, and Theorem K tells us that 5 is finite. 

One of the most convincing examples of the potency of Theorem K arises in 
connection with a family of interesting tiling problems introduced by Hao Wang. 
A tetrad type is a square divided into four parts, each part having a specified 
number in it, such as 



The problem of tiling the plane is to take a finite set of tetrad types, with an 
infinite supply of tetrads of each type, and to show how to place one in each 
square of an infinite plane (without rotating or reflecting the tetrad types) in 
such a way that two tetrads are adjacent only if they have equal numbers where 
they touch. For example, we can tile the plane using the six tetrad types 



in essentially only one way, by repeating the rectangle 



( 3 ) 
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over and over. The reader may easily verify that there is no way to tile the plane 
with the three tetrad types 



Wang’s observation [Scientific American 213,5 (November 1965), 98-106] is 
that if it is possible to tile the upper right quadrant of the plane, it is possible 
to tile the whole plane. This is certainly unexpected, since a method for tiling 
the upper right quadrant involves a “boundary” along the x and y axes, and it 
would seem to give no hint as to how to tile the upper left quadrant of the plane 
(since tetrad types may not be rotated or reflected). We cannot get rid of the 
boundary merely by shifting the upper-quadrant solution down and to the left, 
since it does not make sense to shift the solution by more than a finite amount. 
But Wang s proof runs as follows: The existence of an upper-right-quadrant 
solution implies that there is a way to tile a 2 n x 2 n square, for all n. The set 
of all solutions to the problem of tiling squares with an even number of cells on 
each side forms an oriented tree, if the children of each 2 n x 2 n solution x are 
the possible (2 n + 2) x (2 n + 2) solutions that can be obtained by bordering x. 
The root of this oriented tree is the 0 x 0 solution; its children are the 2 x 2 
solutions, etc. Each node has only finitely many children, since the problem of 
tiling the plane assumes that only finitely many tetrad types are given; hence by 
the infinity lemma there is an infinite path to the root. This means that there 
is a way to tile the whole plane (although we may be at a loss to find it)! 

For later developments in tetrad tiling, see the beautiful book Tilings and 
Patterns by B. Griinbaum and G. C. Shephard (Freeman, 1987), Chapter 11. 

EXERCISES 

1. [lUlf!] The text refers to a set S containing finite sequences of positive integers, 
and states that this set is “essentially an oriented tree.” What is the root of this 
oriented tree, and what are the arcs? 

2. [20] Show that if rotation of tetrad types is allowed, it is always possible to tile 
the plane. 

► 3. [M23] If it is possible to tile the upper right quadrant of the plane when given an 
infinite set of tetrad types, is it always possible to tile the whole plane? 

4. [M25] (H. Wang.) The six tetrad types (2) lead to a toroidal solution to the 
tiling problem, that is, a solution in which some rectangular pattern — namely (3) — 
is replicated throughout the entire plane. 

Assume without proof that whenever it is possible to tile the plane with a finite 
set of tetrad types, there is a toroidal solution using those tetrad types. Use this 
assumption together with the infinity lemma to design an algorithm that, given the 
specifications of any finite set of tetrad types, determines in a finite number of steps 
whether or not there exists a way to tile the plane with these types. 

5. [M 40 ] Show that using the following 92 tetrad types it is possible to tile the plane, 
but that there is no toroidal solution in the sense of exercise 4. 
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To simplify the specification of the 92 types, let us first introduce some notation. 
Define the following “basic codes” : 

« = (1. 2, 1, 2) /9 = (3,4, 2,1) 7 =(2,1, 3, 4) 5 = ( 4, 3, 4, 3) 

a=(Q,D,P,R ) b = ( , ,L,P) c = (U, Q, T, S) d=( , ,S,T) 

N=(Y, ,X, ) J = (D,U, ,X) K = ( , Y, R, L) B = ( , , , ) 

R = ( , ,R,R) L = ( , ,L, L) P = ( , ,P, P) S = (, ,S,S) 

T = ( , ,T,T) X = ( , ,X,X) 

Y = (Y,Y, , ) U=(U,U, , ) D = (D,D, , ) Q = (Q,Q, , ) 

The tetrad types are now 

a{a, b, c, d} [4 types] 

P {Y{B,U,Q}{P,T}, {B, U,D,Q}{P,S,T}, K{B,U,Q}} [21 types] 

7 {{{X, B}{L, P , S, T}, R}{B, Q}, J{L, P , S, T}} [22 types] 

5{X{L,P,S,T}{B,Q}, Y {B, U, Q}{P,T}, N{a,b,c,d}, 

J{L, P, S, T}, K{B , U, Q}, {R, L, P, S , T}{B, U, D, Q}} [45 types] 

These abbreviations mean that the basic codes are to be put together component by 
component and sorted into alphabetic order in each component; thus 

I3Y{B,U,Q}{P,T} 

stands for six types 0 YBP, 0 YUP, 0 YQP, 0YBT, 0 YUT, 0 YQT. The type 0 YQT is 

(3,4,2,l)(y,y, , )(Q,Q, , )( , ,T,T) = (3QY, 4QY, 2T, IT) 

after multiplying corresponding components and sorting into order. 

This is intended to correspond to the tetrad type shown on the right, 
where we use strings of symbols instead of numbers in the four quarters 
of the type. Two tetrad types can be placed next to each other only if 
they have the same string of symbols at the place they touch. 

A /1-tetrad is one that has a 0 in its specification as given above. To get started 
on the solution to this exercise, note that any /3-tetrad must have an a-tetrad to its 
left and to its right, and a (5-tetrad above and below. An aa-tetrad must have 0KB or 
0KU or 0KQ to its right, and then must come an ah-tetrad, etc. 

(This construction is a simplified version of a similar one given by Robert Berger, 
who went on to prove that the general problem in exercise 4, without the invalid 
assumption, cannot be solved. See Memoirs Amer. Math. Soc. 66 (1966).) 

► 6. [ M23 ] (Otto Schreier.) In a famous paper [JVieuw Archief voor Wiskunde (2) 15 
(1927), 212-216], B. L. van der Waerden proved the following theorem: 

If k and m are positive integers, and if we have k sets Si, . . , , Sk of positive integers 
with every positive integer included in at least one of these sets, then at least one 
of the sets Sj contains an arithmetic progression of length m. 

(The latter statement means there exist integers a and (5 > 0 such that a + S, a + 25, 
a + m5 are all in Sj.) If possible, use this result and the infinity lemma to prove 
the following stronger statement: 

If k and m are positive integers, there is a number N such that if we have k sets 
Si,...,S k of integers with every integer between 1 and N included in at least one 
of these sets, then at least one of the sets Sj contains an arithmetic progression of 
length m. 
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► 7. [M30] If possible, use van der Waerden’s theorem of exercise 6 and the infinity 
lemma to prove the following stronger statement: 

If k is a positive integer, and if we have k sets Si,. .. ,S k of integers with every 
positive integer included in at least one of these sets, then at least one of the 
sets Sj contains an infinitely long arithmetic progression. 

► 8. [M3 9] (J. B. Kruskal.) If T and T' are (finite, ordered) trees, let the notation 
T C T' signify that T can be embedded in T', as in exercise 2.3.2-22. Prove that if 
Ti, Th, T 3 , ... is any infinite sequence of trees, there exist integers j < k such that 
Tj C Tk . (In other words, it is impossible to construct an infinite sequence of trees in 
which no tree contains any of the earlier trees of the sequence. This fact can be used 
to prove that certain algorithms must terminate.) 


*2. 3. 4. 4. Enumeration of trees. Some of the most instructive applications of 
the mathematical theory of trees to the analysis of algorithms are connected with 
formulas for counting how many different trees there are of various kinds. For 
example, if we want to know how many different oriented trees can be constructed 
having four indistinguishable vertices, we find that there are just 4 possibilities: 


A 



For our first enumeration problem, let us determine the number a n of 
structurally different oriented trees with n vertices. Obviously, ai = 1. If n > 1, 
the tree has a root and various subtrees; suppose there are j i subtrees with 1 
vertex, j 2 with 2 vertices, etc. Then we may choose j k of the a k possible A:-vertex 
trees in 

f T jk I \ 

V jk J 

ways, since repetitions are allowed (exercise 1.2.6-60), and so we see that 


E 

jl+2j 2 +-=n-l 


a i +j i-l 
3i 


f tin — 1 T jn — 1 
\ jn — 1 



for n > 1. ( 2 ) 


If we consider the generating function A(z) = ^ n a n z", 
that the identity 


1 

(1 ~z r ) a 



with a 0 = 0, we find 


together with ( 2 ) implies 


(1-2)“1(1-Z 2 )«a(l-Z 3 )«3... ■ (3) 

This is not an especially nice form for A(z), since it involves an infinite product 
and the coefficients a\, a 2 , ■ ■ ■ appear on the right-hand side. A somewhat more 
aesthetic way to represent A(z) is given in exercise 1; it leads to a reasonably 
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efficient formula for calculating the values a n (see exercise 2 ) and, in fact, it also 
can be used to deduce the asymptotic behavior of a n for large n (see exercise 4 ). 
We find that 


A(z) = z + z 2 + 2z 3 + 4 z 4 + 9z 5 + 20 z 6 + 48 z 7 + 115 z 8 

+ 286^ 9 + 719^ 10 + 1842Z 11 + • • • . ( 4 ) 

Now that we have essentially found the number of oriented trees, it is quite 
interesting to determine the number of structurally different free trees with n 
vertices. There are just two distinct free trees with four vertices, namely 


and 


(5) 


because the first two and last two oriented trees of ( 1 ) become identical when 
the orientation is dropped. 

We have seen that it is possible to select any vertex X of a free tree and to 
assign directions to the edges in a unique way so that it becomes an oriented tree 
with X as root. Once this has been done, for a given vertex X, suppose there are 
k subtrees of the root X, with «i, s 2 , . . ., Sfc vertices in these respective subtrees. 

Clearly, k is the number of arcs touching X , and Si + s 2 H \- s k — n ~ 1. In 

these circumstances we say that the weight of X is max(s!, s 2 , • • • , s k ). Thus in 
the tree 


B E F 


c 

D 

G 





J K H 


the vertex D has weight 3 (each of the subtrees leading from D has three of the 
nine remaining vertices), and vertex E has weight max(7, 2) = 7. A vertex with 
minimum weight is called a centroid of the free tree. 

Let X and si,s 2 , . . , , s k be as above, and let Tj , Y 2 , . . . , Y k be the roots of 
the subtrees emanating from X. If Y is any node in the Yj subtree, its weight 
must be at least n — s 1 = l + s 2 + - • - + s k , since when Y is the assumed root there 
are at least n - s ± vertices in its subtree containing X. Thus if Y is a centroid 
we have 


weight (X) = max (si, s 2 , • • . , s k ) > weight (Y) > 1 + s 2 H h s k , 

and this is possible only if > s 2 + ■ • ■ + s k . A similar result may be derived if 
we replace Yi by Yj in this discussion. So at most one of the subtrees at a vertex 
can contain a centroid. 

This is a strong condition, for it implies that there are at most two centroids 
in a free tree, and if two centroids exist, they are adjacent. (See exercise 9.) 
Conversely, if > s 2 + f s k , there is a centroid in the Yi subtree, since 

weight (Yi) < max («x — 1, 1 + s 2 + b s k ) < si — weight (X), 
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and the weight of all nodes in the Y 2 ,...,Y k subtrees is at least Si + 1. We have 
proved that the vertex X is the only centroid of a free tree if and only if 

sj < Si + ■ ■ ■ + s k - sj, for 1 < j < k. (7) 

Therefore the number of free trees with n vertices, having only one centroid, 
is the number of oriented trees with n vertices minus the number of such oriented 
trees violating condition (7); the latter consist essentially of an oriented tree with 
Sj vertices and another oriented tree with n — Sj < Sj vertices. The number with 
one centroid therefore comes to 

a n ~ dl&n— 1 d 2 (l n — 2 — • • • — a^ n j2\ a [n/2] • (8) 

A free tree with two centroids has an even number of vertices, and the weight of 
each centroid is n/2 (see exercise 10). So if n = 2m, the number of bicentroidal 
free trees is the number of choices of 2 things out of a m with repetition, namely 

C” 2 +1 )' 

To get the total number of free trees, we therefore add \a n/2 (a n / 2 + 1) to (8) 
when n is even. The form of Eq. (8) suggests a simple generating function, and 
indeed, we find without difficulty that the generating function for the number of 
structurally different free trees is 

F(z) = A(z)-±A(z) 2 +±A(z 2 ) 

= z + z 2 + z 3 + 2z 4 + 3 z 5 + 6 2 6 + Il 2 7 + 23z 8 

+ 47 z 9 + 106 z 10 + 235 z 11 + ■■■. ( 9 ) 

This simple relation between F(z ) and A(z) is due primarily to C. Jordan, who 
considered the problem in 1869. 

Now let us turn to the question of enumerating ordered trees, which are our 
principal concern with respect to computer programming algorithms. There are 
five structurally different ordered trees with four vertices: 


\A\k* 


(io) 


The first two are identical as oriented trees, so only one of them appeared in (i) 
above. 

Before we examine the number of different ordered tree structures, let us 
first consider the case of binary trees, since this is closer to the actual computer 
representation and it is easier to study. Let b n be the number of different binary 
trees with n nodes. From the definition of binary tree it is apparent that b 0 = 1, 
and for n > 0 the number of possibilities is the number of ways to put a binary 
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tree with k nodes to the left of the root and another with n-l-k nodes to the 
right. So 

b n = bob n -i + b\b n -2 + • • • + b n -ibo, n > 1. (n) 

From this relation it is clear that the generating function 

-B(-z) — bo + b\z + b2Z 2 + • • • 

satisfies the equation 

zB(z ) 2 = B(z) - 1. (12) 

Solving this quadratic equation and using the fact that B( 0) = 1, we obtain 


fc> 0 


*<*)-£( 

= £ 


-4 z y 


(-4z) n 
n + 1 


i>0 


2n 

n J n + 1 


= 1 + z + 2z 2 + 5z 3 + 14z 4 + 42z 5 + 132z 6 + 429z 7 


1430z 8 + 4862z y + 16796z 


10 


(See exercise 1.2.6-47.) The desired answer is therefore 

1 /2 n 

n + 1 \ n 


bn — 


A) 


(14) 


By Stirling’s formula, this is asymptotically \ n jnyjlvn + 0(4 n n -5 / 2 ). Some 
important generalizations of Eq. (14) appear in exercises 11 and 32. 

Returning to our question about ordered trees with n nodes, we can see that 
this is essentially the same question as the number of binary trees, since we have 
a natural correspondence between binary trees and forests, and a tree minus its 
root is a forest. Hence the number of (ordered) trees with n vertices is 6 n -i, the 
number of binary trees with n — 1 vertices. 

The enumerations performed above assume that the vertices are indistin- 
guishable points. If we label the vertices 1, 2, 3, 4 in (1) and insist that 1 is to 
be the root, we now get 16 different oriented trees: 

1a 1a lA 1 a lA lA 1 

2 
3 



1] 

ll 

ll 

ll 

it 

, , 

< > 

^ , 

, y 


' 2 

3 

■ 3 

4 • 

4 , 

< j 

, 

^ / 

^ y 

, , 

' 4' 

2 

’ 4 

• 2 1 

3" 




, > 

, , 

• 3' 

4 

' 2 

3' 

2" 



A 

2 3 4 


U5) 


4 
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The question of enumeration for labeled trees is clearly quite different from the 
one solved above. In this case it can be rephrased as follows: “Consider drawing 
three lines, pointing from each of the vertices 2 , 3 , and 4 to another vertex; 
there are three choices of lines emanating from each vertex, so there are 3 3 = 27 
possibilities in all. How many of these 27 ways will yield oriented trees with 1 
as the root? The answer, as we have seen, is 16. A similar reformulation of the 
same problem, this time for the case of n vertices, is the following: “Let f(x) be 
an integer-valued function such that /( 1) = 1 and 1 < /( x) < n for all integers 
1 < X < n. We call / a tree mapping if that is, /(/(••• (f(x)) ■■■ )) 

iterated n times, equals 1, for all x. How many tree mappings are there?” This 
problem comes up, for example, in connection with random number generation. 
We will find, rather surprisingly, that on the average exactly one out of every n 
such functions / is a tree mapping. 

The solution to this enumeration problem can readily be derived using the 
general formulas for counting subtrees of graphs that have been developed in 
previous sections (see exercise 12 ). But there is a much more informative way 
to solve the problem, one that gives us a new and compact manner to represent 
oriented tree structure. 

Suppose that we’ve been given an oriented tree with vertices {1,2, ...,n} 
and with n - 1 arcs, where the arcs go from j to f(j) for all j except the root. 
There is at least one terminal (leaf) vertex; let V) be the smallest number of a 
leaf. If n > 1, write down /(Vi) and delete both Vi and the arc V) -a /(Vi) 
from the tree; then let V 2 be the smallest number whose vertex is terminal in 
the resulting tree. If n > 2 , write down f(V 2 ) and delete both V 2 and the arc 
V 2 ->■ /O' 2 ) from the tree; and proceed in this way until all vertices have been 
deleted except the root. The resulting sequence of n — 1 numbers, 

f( v i), f(V 2 ), /04_!), 1 <f{Vj)<n, ( 16 ) 

is called the canonical representation of the original oriented tree. 

For example, the oriented tree 



with 10 vertices has the canonical representation 1, 3, 10, 5, 10, 1, 3, 5, 3. 

The important point here is that we can reverse this process and go from 
any sequence of n - 1 numbers ( 16 ) back to the oriented tree that produced it. 
For if we have any sequence x ± , x 2 , . . . , z n _! of numbers between 1 and n, let 
V 1 be the smallest number that does not appear in the sequence x ly . . . , ar n _ i; 
then let V 2 be the smallest number / V) that does not appear in the sequence 
x 2j , x n —i , and so on. After obtaining a permutation V) V)> . . . V n of the integers 
{1, 2, . . . , n} in this way, draw arcs from vertex V) to vertex x } , for 1 < j < n. 
This gives a construction of a directed graph with no oriented cycles, and by 
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exercise 2 . 3 . 4 . 2-7 it is an oriented tree. Clearly, the sequence x x ,x 2 , . . . ,x n ~ x is 
the same as the sequence (16) for this oriented tree. 

Since the process is reversible, we have obtained a one-to-one correspondence 
between (n — l)-tuples of numbers {1,2, ...,n} and oriented trees on these 
vertices. Hence there are n"^ 1 distinct oriented trees with n labeled vertices. If 
we specify that one vertex is to be the root, there is clearly no difference between 
one vertex and another, so there are n n “ 2 distinct oriented trees on {1, 2 , . . . , n} 
having a given root. This accounts for the 16 = 4 4-2 trees in (15). From this 
information it is easy to determine the number of free trees with labeled vertices 
(see exercise 22 ). The number of ordered trees with labeled vertices is also easy to 
determine, once we know the answer to that problem when no labels are involved 
(see exercise 23 ). So we have essentially solved the problems of enumerating the 
three fundamental classes of trees, with both labeled and unlabeled vertices. 

It is interesting to see what would happen if we were to apply our usual 
method of generating functions to the problem of enumerating labeled oriented 
trees. For this purpose we would probably find it easiest to consider the quantity 
r(n,q), the number of labeled directed graphs with n vertices, with no oriented 
cycles, and with one arc emanating from each of q designated vertices. The 
number of labeled oriented trees with a specified root is therefore r(n, n — 1 ). In 
this notation we find by simple counting arguments that, for any fixed integer to, 

r ( n 1 l) - yiQ V(m + fc, k) r(n — m — k, q-k), if 0 <m<n-q, (18) 

k 

r(n, q) = J 2 [l) r ( n -l, <l-k), if q = n - l. (19) 

k 

The first of these relations is obtained if we partition the undesignated vertices 
into two groups A and B , with m vertices in A and n - q - m vertices in B\ 
then the q designated vertices are partitioned into k vertices that begin paths 
leading into A , and q — k vertices that begin paths leading into B. Relation (19) 
is obtained by considering oriented trees in which the root has degree k. 

The form of these relations indicates that we can work profitably with the 
generating function 


G m (z) = r(m, 0) + r(m + 1, 1 )z + 


r(m + 2, 2 )z 2 

2 ! 


= £ 


r(k + to, k)z k 

k\ 


In these terms Eq. (18) says that G n _,(z) = G m (z)G n _ q _ m {z), and therefore by 
induction on to, we find that G rn (z) ~ G x {z) m . Now from Eq. (19), we obtain 


Gi(z) = J 2 

n> 1 


r(n, n 
(n 


1 )z n ~ 1 

T)!~ 


££ 

k>0 n> 1 


r(n — 1, n — 1 — k)z n 1 
k\ (n — 1 — k)\ 


= £&*(*) = £ 

k>0 k > 0 


{zCh[z)) k 


— e zGi(z). 


k\ 
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In other words, putting Gi(z) = w, the solution to our problem comes from 
the coefficients of the solution to the transcendental equation 


w = e 


ZW 


(20) 


This equation can be solved with the use of Lagrange’s inversion formula: 
z = C/ /(C) implies that 


< = £ 

n>l 



(n-l) 

n 


(o), 


(21) 


where g n ( () = f(() n , when / is analytic in the neighborhood of the origin, and 
/( 0) / 0 (see exercise 4.7-16). In this case, we may set C = zw, /(C) = e c , and 
we deduce the solution 


w = 


£ 

n> 0 


(n+ iy 


(22) 


in agreement with the answer obtained above. 

G. N. Raney has shown that we can extend this method in an important 
way to obtain an explicit power series for the solution to the considerably more 
general equation 

w = y x e ZlW + y 2 e Z2W + f y s e z “ w , 

solving for w in terms of a power series in y if . . . , y s and z 1: ...,z s . For this 
generalization, let us consider s-dimensional vectors of integers 


n (u-i , 77-2 , . . . , TL S ) , 

and let us write for convenience 


5Cn — ni + ri 2 + • • ■ + n s . 

Suppose that we have s colors C\, C 2 , . . . ,C S , and consider directed graphs 
in which each vertex is assigned a color; for example, 



Let r(n, q) be the number of ways to draw arcs and to assign colors to the 
vertices {1, 2, . . . , n}, such that 

i) for 1 < i < s there are exactly n; vertices of color Ci (hence n = En); 

ii) there are q arcs, one leading from each of the vertices {1,2 ,q}; 

iii) for 1 < i < s there are exactly arcs leading to vertices of color C t (hence 

< 1 = Eq); 

iv) there are no oriented cycles (hence q < n, unless q = n — 0). 

Let us call this an (n,q)-construction. 
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For example, if C\ — red, C 2 — yellow, and C 3 = blue, then ( 23 ) shows 
a ((3, 2, 2), ( 1 , 2, 2)) -construction. When there is only one color, we have the 
oriented tree problem that we have already solved. Raney’s idea is to generalize 
the one-dimension construction to s dimensions. 

Let n and q be fixed s-place vectors of nonnegative integers, and let n — X!n, 
q — £q. For each (n,q)-construction and each number k, 1 < k < n, we will 
define a canonical representation consisting of four things: 

a) a number t, with q < t < n; 

b) a sequence of n colors, with n % of color C*; 

c) a sequence of q colors, with g* of color C. t \ 

d) for 1 < i < s, a sequence of q t elements of the set { 1 , 2 , ... , n,}. 

The canonical representation is defined thus: First list the vertices { 1 , 2, . . . , q} 
in the order V 1; V 2 , . . . , V ? of the canonical representation of oriented trees (as 
given above), and then write below vertex Vj the number f(Vj) of the vertex on 
the arc leading from Vj. Let t = f(V q ); and let the sequence (c) of colors be the 
respective colors of the vertices /(Vi), . . . , f(V q ). Let the sequence (b) of colors 
be the respective colors of the vertices k, k + 1, . . . , n, 1, . . , , k - 1. Finally, let 
the ith sequence in (d) be xu,x i 2 , ■ ■ ■ , x iqi , where a \j = m if the jth Cj-colored 
element of the sequence f(Vi),...,f(V q ) is the mth Cj-colored element of the 
sequence k, k + 1 , . . . , n, 1 , . . . , k - 1 . 

For example, consider construction ( 23 ) and let k = 3. We start by listing 
Vi, . . . , V 5 and /(Vi), . . . , /(V 5 ) below them as follows: 

1 2 4 5 3 
7 6 3 3 6 

Hence t = 6 , and sequence (c) represents the respective colors of 7, 6 , 3, 3, 6 , 
namely red, yellow, blue, blue, yellow. Sequence (b) represents the respective 
colors of 3, 4, 5, 6 , 7, 1, 2, namely blue, yellow, red, yellow, red, blue, red. 
Finally, to get the sequences in (d), proceed as follows: 


color 

elements this color 
in 3, 4, 5, 6 , 7, 1, 2 

elements this color 
in 7, 6 , 3, 3 , 6 

encode column 3 
by column 2 

red 

5,7,2 

7 

2 

yellow 

4,6 

6, 6 

2,2 

blue 

3,1 

3,3 

1,1 


Hence the (d) sequences are 2; 2,2; and 1, 1. 

From the canonical representation, we can recover both the original (Re- 
construction and the number k as follows: From (a) and (c) we know the color of 
vertex t. The last element of the (d) sequence for this color tells us, in conjunction 
with (b), the position of t in the sequence k, . . . , n, 1 , . . . , k — 1 ; hence we know 
k and the colors of all vertices. Then the subsequences in (d) together with 
(b) and (c) determine f(Vi),f(V 2 ),...,f(V q ), and finally the directed graph is 
reconstructed by locating Vi, ... ,V q as we did for oriented trees. 
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The reversibility of this canonical representation allows us to count the 
number of possible (n,q)-constructions, since there are n — q choices for (a), 
and the multinomial coefficient 

n 

Til , , n s 

choices for (b), and 

( 5 

choices for (c), and n\' nf . . . nj 3 choices for (d). Dividing by the n choices for k, 
we have the general result 


~(n,q) = 


n — q 


q'- 


T n?nf...n! 


n ni! ...n s \ qi\ . . . q s \ 
Furthermore, we can derive analogs of Eqs. (18) and (19): 


(24) 


~( n ,q) = E 

k,t 

£(t-k)=m 


£q 

£k 


r(t,k)r(n - t,q - k) if 0 < m < E(n - q), (25) 


with the convention that r(0,0) = 1, and r(n,q) = 0 if any Hi or q, is negative 
or if q > n; 

if£n=l + Eq, (26) 

where e t is the vector with 1 in position i and zeros elsewhere. Relation (25) 
is based on breaking the vertices {q + 1, . . . , n} into two parts having m and 
n — q — m elements, respectively; the second relation is derived by removing 
the unique root and considering the remaining structure. We now obtain the 
following result: 


Theorem R (George N. Raney, Canadian J. Math. 16 (1964), 755-762). Let 


w 


E 

n.q 

£(n-q) = l 


r( n.q) 

(£q)! 


vT- 


.91 


2 


Qs 
s > 


(27) 


where r(n, q) is defined by (24), and where n, q are s-dimensional integer vectors. 
Then w satisfies the identity 


w = yi e ZlW + y 2 e Z2W + ■ ■ ■ + y s e z 


(28) 


Proof. By (25) and induction on m, we find that 


w 


m 


£ 

n,q 

E(n-q)=m 


rijnqf 

(£q )! 


n 1 

Vi 


y n s Z Q 1 

• Vs z l 


. . Z 


Qs 

s * 


(29) 
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Now by ( 26 ), 


w 


EE E 

i = l k n .q 

E(n-q) = l 


r(n - ej, q - fcej 

kl (Eq - k)\ 


Vi 1 ■ ■ ■ y n s’ 4 1 ■ ■ ■ 4‘ 


= EEh^ e 


1=1 k 


r A n - z ,, 

y 1 • • • Vs z i - • • z s 


n,q 

E(n-q )=k 


(Eq)! 


“EEs' 


i=l /e 




The special case where s — 1 and z x = 1 in ( 27 ) and ( 28 ) is especially 
important in applications, so it has become known as the “tree function” 


n n- l 

ny) = Y.—ry n =ye T{v) - (30) 

n> 1 

See Corless, Gonnet, Hare, Jeffrey, and Knuth, Advances in Computational 
Math. 5 (1996), 329-359, for a discussion of this function’s history and some 
of its remarkable properties. 

A survey of enumeration formulas for trees, based on skillful manipulations 
of generating functions, has been given by I. J. Good [Proc. Cambridge Philos. 
Soc. 61 (1965), 499-517; 64 (1968), 489]. More recently, a mathematical theory 
of species developed by Andre Joyal [Advances in Math. 42 (1981), 1-82] has 
led to a high-level viewpoint in which algebraic operations on generating func- 
tions correspond directly to combinatorial properties of structures. The book 
Combinatorial Species and Tree-like Structures by F. Bergeron, G. Labelle, and 
P. Leroux (Cambridge Univ. Press, 1998), presents numerous examples of this 
beautiful and instructive theory, generalizing many of the formulas derived above. 


EXERCISES 

1 . [M20] (G. Polya.) Show that 

A(z) = z ■ exp (A{z) + \A(z 2 ) + | A(z 3 ) -I ). 

[Hint: Take logarithms of ( 3 ).] 

2 . [ HM24 ] (R- Otter.) Show that the numbers a n satisfy the following condition: 


where 


Ufln+l — &lS n i -|- 2d2Sn2 4 “ * * ' + TlCL n S 

Snk — ^ ] Q n +l—jk. 

l<j<n/fc 


(These formulas are useful for the calculation of the a„, since s n k = S(n-fc)fc +a*i+ i-k-) 
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3. [M40] Write a computer program that determines the number of (unlabeled) free 
trees and of oriented trees with n vertices, for n < 100. (Use the result of exercise 2.) 
Explore arithmetical properties of these numbers; can anything be said about their 
prime factors, or their residues modulo p? 

4. [HM39] (G. Polya, 1937.) Using complex variable theory, determine the asymp- 
totic value of the number of oriented trees as follows: 

a) Show that there is a real number a between 0 and 1 for which A(z ) has radius of 
convergence a and A(z) converges absolutely for all complex 2 such that \z\ < a, 
having maximum value A(a ) = a < 00 . [Hint: When a power series has nonnega- 
tive coefficients, it either is entire or has a positive real singularity; and show that 
A{z)f z is bounded as z — > a — , by using the identity in exercise 1.] 

b) Let 

F(z, w) = exp ( zw + | A(z 2 ) + \A(z 3 ) + ■■■) -w. 

Show that in a neighborhood of ( z,w ) = (a, a/a), F(z,w) is analytic in each 
variable separately. 

c) Show that at the point ( 2 , w) = (a, a/a), we have dF/dw = 0; hence a = 1 . 

d) At the point ( z,w ) = (a, 1/a) show that 


dF 

dz 


= P = a~ 2 + Y j a k ~ 2 A'{a k ), 

k> 2 


and 


d 2 F _ 
dw 2 


e) When \z\ = a and 2 / a, show that dF/dw / 0; hence A(z) has only one 
singularity on \z\ = a. 

f) Prove that there is a region larger than \z\ < a in which 

2 = a ~ V 2 /3(l “ z /&) + (1 - z/a)R(z), 


where R(z) is an analytic function of \/ z — a. 

g) Prove that consequently 


Un = ^7^ + 0(n- 5/2 a-). 

[Note: 1/a « 2.955765285652, and a^/3/2n « 0.439924012571.] 

5. [M25] (A. Cayley.) Let c n be the number of (unlabeled) oriented trees having 
n leaves (namely, vertices with in-degree zero) and having at least two subtrees at 
every other vertex. Thus c 3 = 2, by virtue of the two trees 



Find a formula analogous to ( 3 ) for the generating function 

C{z) = Y.CnZ n . 

n 

6 . [M25] Let an “oriented binary tree” be an oriented tree in which each vertex has 
in-degree two or less. Find a reasonably simple relation that defines the generating 
function G(z) for the number of distinct oriented binary trees with n vertices, and find 
the first few values. 
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7. [HM40] Obtain asymptotic values for the numbers of exercise 6. (See exercise 4.) 

8. [20] According to Eq. (g), there are six free trees with six vertices. Draw them, 
and indicate their centroids. 

9. [M20] From the fact that at most one subtree of a vertex in a free tree can contain 
a centroid, prove that there are at most two centroids in a free tree; furthermore if there 
are two, then they must be adjacent. 

► 10. [M2 2] Prove that a free tree with n vertices and two centroids consists of two free 
trees with n/2 vertices, joined by an edge. Conversely, if two free trees with m vertices 
are joined by an edge, we obtain a free tree with 2 m vertices and two centroids. 

► 11. [M28] The text derives the number of different binary trees with n nodes (14). 
Generalize this to find the number of different t-ary trees with n nodes. (See exercise 
2.3. 1 35; a t-ary tree is either empty or consists of a root and t disjoint t-ary trees.) 
Hint: Use Eq. (21) of Section 1.2.9. 

12. [M20] Find the number of labeled oriented trees with n vertices by using deter- 
minants and the result of exercise 2.3.4.2-19. (See also exercise 1.2.3-36.) 

13. [15] What oriented tree on the vertices {1, 2, . . . , 10} has the canonical represen- 
tation 3, 1, 4, 1, 5, 9, 2, 6, 5? 

14. [10] True or false: The last entry, f(V n - 1), in the canonical representation of an 
oriented tree is always the root of that tree. 

15. [21] Discuss the relationships that exist (if any) between the topological sort 
algorithm of Section 2.2.3 and the canonical representation of an oriented tree. 

16. [25] Design an algorithm (as efficient as possible) that converts from the canonical 
representation of an oriented tree to a conventional computer representation using 
PARENT links. 

►17. [M26] Let f(x) be an integer- valued function, where 1 < f(x ) < m for all integers 
1 < m. Define x = y if f [r \x) = f [s \y) for some r, s > 0, where f l ° ] (x) = x and 

f l ' + 1 ( x ) = f(f lr] i x ))- By using methods of enumeration like those in this section, 
show that the number of functions such that x = y for all x and y is m m ~ 1 Q(m), 
where Q(m) is the function defined in Section 1.2.11.3. 

18. [24] Show that the following method is another way to define a one-to-one cor- 
respondence between (n — l)-tuples of numbers from 1 to n and oriented trees with 
n labeled vertices: Let the leaves of the tree be I j , . . . , l in ascending order. Let 
(!t> 1 fc+i, VI+2, • • ■ , Uj) be the path from 1 _ 1 to the root, and write down the vertices 
^ <z> • ■ • > ^4+2, V k+X . Then let (Vh, V q+X , V q+ 2 , . . . , V r ) be the shortest oriented path from 
V 2 such that V r has already been written down, and write down V ry . . . ,V q+2 ,V q+1 . 
Then let (V 3 , V r+1 , . . . , U s ) be the shortest oriented path from V 3 such that V s has 
already been written, and write V s , . . . , 14+1 ; and so on. For example, the tree (17) 
would be encoded as 3, 1, 3, 3, 5, 10, 5, 10, 1. Show that this process is reversible, 
and in particular, draw the oriented tree with vertices {1,2 ,..., 10} and representation 
3, 1, 4, 1, 5, 9, 2, 6, 5. 

19. [M24] How many different labeled, oriented trees are there having n vertices, k of 
which are leaves (have in-degree zero)? 

20. [M 2 4 ] (J. Riordan.) How many different labeled, oriented trees are there having 
n vertices, k 0 of which have in-degree 0, k x have in-degree 1 , k 2 have in-degree 2, . . . ? 
(Note that necessarily k 0 + h + k 2 + • • • = n, and k x + 2fe 2 + 3k 3 + ■ ■ ■ = n - 1.) 
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► 21. [M21] Enumerate the number of labeled oriented trees in which each vertex has 
in-degree zero or two. (See exercise 20 and exercise 2.3-20.) 

22. [ M20 ] How many labeled free trees are possible with n vertices? (In other words, if 
we are given n vertices, there are 2(2) possible graphs having these vertices, depending 
on which of the (”) possible edges are incorporated into the graph; how many of these 
graphs are free trees?) 

23 . [M21] How many ordered trees are possible with n labeled vertices? (Give a 
simple formula involving factorials.) 

24 . [ M16 ] All labeled oriented trees with vertices 1, 2, 3, 4 and with root 1 are shown 
i n (is)- How many would there be if we listed all labeled ordered trees with these 
vertices and this root? 

25 . [M20] What is the value of the quantity r(n,q) that appears in Eqs. (18) and 
(19)? (Give an explicit formula; the text only mentions that r(n, n - 1) = n n ~ 2 .) 

26 . [20] In terms of the notation at the end of this section, draw the ((3, 2, 4), (1.4, 2))- 
construction, analogous to (23), and find the number k that corresponds to the canon- 
ical representation having t = 8, the sequences of colors “red, yellow, blue, red, yellow, 
blue, red, blue, blue” and “red, yellow, blue, yellow, yellow, blue, yellow”, and the 
index sequences 3; 1, 2, 2, 1; 2, 4. 

► 27 . [ M28 ] Let Ui, U 2 , ... , U p , . . . , U g ; Vi, V 2 , . . . , V r be vertices of a directed graph, 
where 1 < p < q. Let / be any function from the set {p + 1, . . . , q] into the set 
{1,2,..., r}, and let the directed graph contain exactly q-p arcs, from U k to V f ( k) for 
p < k < q. Show that the number of ways to add r additional arcs, one from each of the 
U’s to one of the U’s, such that the resulting directed graph contains no oriented cycles, 
is q r ~ 1 p. Prove this by generalizing the canonical representation method; that is, set 
up a one-to-one correspondence between all such ways of adding r further arcs and the 
set of all sequences of integers cn, a 2 , . . . , a r , where 1 < a k < q for 1 < k < r, and 
1 < a r < p. 

28 . [M22] ( Bipartite trees.) Use the result of exercise 27 to enumerate the number 
of labeled free trees on vertices Ui, ..., U m , Vi, ..., V n , such that each edge joins 
Uj to Vk for some j and k. 

29 . [ HM26 ] Prove that if E k (r,t) = r(r + kt) k ~ 1 /k\, and if zx t = \nx, then 

x r = Y J E k {r,t)z k 

k> 0 

for fixed t and for sufficiently small \z\ and \x - 1|. [Use the fact that G m {z) = G 1 {z) m 
in the discussion following Eq. (19).] In this formula, r stands for an arbitrary real 
number. [Note: As a consequence of this formula we have the identity 

n 

E k (r,t)E n -k{s,t ) = E n (r + s, t); 

k- 0 

this implies Abel’s binomial theorem, Eq. (16) of Section 1.2.6. Compare also Eq. (30) 
of that section.] 

30. [ M23 ] Let n,x,y,zi,...,z n be positive integers. Consider asetofx + y + 2 !-)- 

• • • + z n + n vertices rj, Sj k , tj (1 < i < x + y, 1 < j < n, 1 < k < Zj), in which arcs 
have been drawn from Sj k to tj for all j and k. According to exercise 27, there are 
(x + y)(x + y + zi + 1- z n ) n ~ 1 ways to draw one arc from each of t\, . . . , t n to other 
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vertices such that the resulting directed graph contains no oriented cycles. Use this 
fact to prove Hurwitz’s generalization of the binomial theorem: 

J2x{x + e 1 z 1 + --- + e„z n y i+ - +e «- 1 y(y + (l- ei )z 1 + --- + (l-e n )z n ) n - 1 ~ ei 

= (x + y)(x + y + zi + --- + z n ) n ~ 1 , 
where the sum is over all 2 n choices of ei, . . . , e„ equal to 0 or 1. 

31. [M24] Solve exercise 5 for ordered trees; that is, derive the generating function for 
the number of unlabeled ordered trees with n terminal nodes and no nodes of degree 1. 

32. [M3 7] (A. Erdelyi and I. M. H. Etherington, Edinburgh Math. Notes 32 (1941), 
7-12.) How many (ordered, unlabeled) trees are there with no nodes of degree 0, n j of 
degree 1, . . . , n m of degree m, and none of degree higher than m? (An explicit solution 
to this problem can be given in terms of factorials, thereby considerably generalizing 
the result of exercise 11.) 

► 33. [M28] The text gives an explicit power series solution for the equation w = 
2/ie' lu + •■• + y r e ZrW 1 based on enumeration formulas for certain oriented forests. 
Similarly, show that the enumeration formula of exercise 32 leads to an explicit power 
series solution to the equation 

w = ziw ei + z 2 w e 2 + 1- z r w er , 

expressing w as a power series in zi,...,z r . (Here e.i,...,e r are fixed nonnegative 
integers, at least one of which is zero.) 

2. 3. 4. 5. Path length. The concept of the “path length” of a tree is of great 
importance in the analysis of algorithms, since this quantity is often directly 
related to the execution time. Our primary concern is with binary trees, since 
they are so close to actual computer representations. 

In the following discussion we will extend each binary tree diagram by adding 
special nodes wherever a null subtree was present in the original tree, so that 


becomes 


The latter is called an extended binary tree. After the square-shaped nodes have 
been added in this way, the structure is sometimes more convenient to deal with, 
and we shall therefore meet extended binary trees frequently in later chapters. 
It is clear that every circular node has two children and every square node has 
none. (Compare with exercise 2.3-20.) If there are n circular nodes and s square 
nodes, we have n + s — 1 edges (since the diagram is a free tree); counting another 
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way, by the number of children, we see that there are 2 n edges. Hence it is clear 
that 

s-n+ 1 ; ( 2 ) 

in other words, the number of “external” nodes just added is one more than 
the number of “internal” nodes we had originally. (For another proof, see 
exercise 2.3.1-14.) Formula ( 2 ) is correct even when n = 0 . 

Assume that a binary tree has been extended in this way. The external path 
length of the tree , £, is defined to be the sum — taken over all external (square) 
nodes — of the lengths of the paths from the root to each node. The internal 
path length , 7, is the same quantity summed over the internal (circular) nodes. 
In ( 1 ) the external path length is 

£ = 3 + 3 + 2 + 3 + 4 + 4 + 3 + 3 = 25, 

and the internal path length is 

7 = 2 + 1+0 + 2 + 3 + 1 + 2 = 11. 

These two quantities are always related by the formula 

£ = 7 + 2 n, ( 3 ) 

where n is the number of internal nodes. 

To prove formula ( 3 ), consider deleting an internal node V at a distance k 
from the root, where both children of V are external. The quantity £ goes down 
by 2 (k + 1), since the children of V are removed, then it goes up by k, since V 
becomes external; so the net change in £ is —k — 2. The net change in 7 is —k, 
so ( 3 ) follows by induction. 

It is not hard to see that the internal path length (and hence the external 
path length also) is greatest when we have a degenerate tree with linear structure; 
in that case the internal path length is 

(n - 1 ) + (n - 2 ) H f 1 + 0 = U U . 

It can be shown that the “average” path length over all binary trees is essentially 
proportional to n\fn (see exercise 5). 

Consider now the problem of constructing a binary tree with n nodes that 
has minimum path length. Such a tree will be important, since it will minimize 
the computation time for various algorithms. Clearly, only one node (the root) 
can be at zero distance from the root; at most two nodes can be at distance 1 
from the root, at most four can be 2 away, and so on. Therefore the internal 
path length is always at least as big as the sum of the first n terms of the series 

0; 1) 1) 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, . . . . 

This is the sum Sfc=il_lgkJ> which we know from exercise 1.2.4-42 is 

(n+ l)q - 2 q+1 + 2 , <7 = |.lg(n + 1 )J. 


(4) 
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The optimum value ( 4 ) is nlgn+0(n), since q — lgn+0(l); it is clearly achieved 
in a tree that looks like this (illustrated for n = 12 ): 


(5) 

A tree such as ( 5 ) is called the complete binary tree with n internal nodes. 
In the general case we can number the internal nodes 1, 2, . . . , n; this numbering 
has the useful property that the parent of node k is node [fc/ 2 j, and the children 
of node k are nodes 2k and 2k + 1 . The external nodes are numbered n + 1 
through 2n + 1 , inclusive. 

It follows that a complete binary tree may simply be represented in sequen- 
tial memory locations, with the structure implicit in the locations of the nodes 
(not in links). The complete binary tree appears explicitly or implicitly in many 
important computer algorithms, so the reader should give it special attention. 

These concepts have important generalizations to ternary, quaternary, and 
higher-order trees. We define a t-ary tree as a set of nodes that is either empty 
or consists of a root and t ordered, disjoint t-ary trees. (This generalizes the 
definition of binary tree in Section 2.3.) The complete ternary tree with 12 
internal nodes is 



It is easy to see that the same construction works for any t >2. In the complete 
t-ary tree with internal nodes ( 1 , 2 ,..., n}, the parent of node k is node 

L(* + t-2)/tJ = f(fc-l)/tl, 

and the children of node k are 

t(fc — l) + 2, t(k — 1) + 3, ..., tjfe + l. 
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This tree has the minimum internal path length among all t-ary trees with n 
internal nodes; exercise 8 proves that its internal path length is 

( n + yZTj) 1 !- q = l lo gt((* - !>+ !)J- (7) 

These results have another important generalization if we shift our point 
of view slightly. Suppose that we are given m real numbers Wi, w 2 , ■ • ■ , w m ; 
the problem is to find an extended binary tree with m external nodes, and to 
associate the numbers wi, . . . , w m with these nodes in such a way that the sum 
Y2 wjlj is minimized, where lj is the length of path from the root and the sum is 
taken over all external nodes. For example, if the given numbers are 2, 3, 4, 11, 
we can form extended binary trees such as these three: 



Here the “weighted” path lengths Y2 w jh are 34, 53, and 40, respectively. (There- 
fore a perfectly balanced tree does not give the minimum weighted path length 
when the weights are 2, 3, 4, and 11, although we have seen that it does give the 
minimum in the special case W\ = W 2 = • • • = w m = 1.) 

Several interpretations of weighted path length arise in connection with 
different computer algorithms; for example, we can apply it to the merging of 
sorted sequences of respective lengths w±, W2, ■ ■ ■ , w m (see Chapter 5). One of 
the most straightforward applications of this idea is to consider a binary tree 
as a general search procedure, where we start at the root and then make some 
test; the outcome of the test sends us to one of the two branches, where we may 
make farther tests, etc. For example, if we want to decide which of four different 
alternatives is true, and if these possibilities will be true with the respective 
probabilities ^ ^ , 4L , and li ; a tree that minimizes the weighted path length 
will constitute an optimal search procedure. [These are the weights shown in (8), 
times a scale factor.] 

An elegant algorithm for finding a tree with minimum weighted path length 
was discovered by D. Huffman [Proc. IRE 40 (1952), 1098-1101]: First find the 
two w ' s of lowest value, say W\ and W2 ■ Then solve the problem for m — 1 weights 
w 1 + jc’ 2 , w. 3 , ■ • . , w m , and replace the node 


W1 + W2 


( 9 ) 
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in this solution by 



(10) 


As an example of Huffman’s method, let us find the optimal tree for the 
weights 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41. First we combine 2 + 3, 
and look for the solution to 5, 5, 7, . . . , 41; then we combine 5 + 5, etc. The 
computation is summarized as follows: 


5 

7 

11 

13 

17 

19 

23 

29 

31 

37 

41 

5 

7 

11 

13 

17 

19 

23 

29 

31 

37 

41 

IQ 

7 

11 

13 

17 

19 

23 

29 

31 

37 

41 


17 

n 

13 

17 

19 

23 

29 

31 

37 

41 


17 


24 

17 

19 

23 

29 

31 

37 

41 




24 

34 

19 

23 

29 

31 

37 

41 




24 

34 


42 

29 

31 

37 

41 





34 


42 

53 

31 

37 

41 







42 

53 

65 

37 

41 







42 

53 

65 


78 








95 

65 


78 








95 



143 











238 


Therefore the following tree corresponds to Huffman’s construction: 



(The numbers inside the circular nodes show the correspondence between this 
tree and our computation; see also exercise 9.) 

It is not hard to prove that this method does in fact minimize the weighted 
path length, by induction on to. Suppose we have «+ < w 2 < < ■ ■ ■ < w m , 

where m > 2, and suppose that we are given a tree that minimizes the weighted 
path length. (Such a tree certainly exists, since only finitely many binary trees 
with to terminal nodes are possible.) Let V be an internal node of maximum 
distance from the root. If w x and w 2 are not the weights already attached to the 
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children of V, we can interchange them with the values that are already there; 
such an interchange does not increase the weighted path length. Thus there is 
a tree that minimizes the weighted path length and contains the subtree ( 10 ). 
Now it is easy to prove that the weighted path length of such a tree is minimized 
if and only if the tree with ( 10 ) replaced by ( 9 ) has minimum path length for 
the weights wi + w%, w 3 , . . . , w rn . (See exercise 9.) 

Every time this construction combines two weights, they are at least as big as 
the weights previously combined, if the given w t were nonnegative. This means 
that there is a neat way to find Huffman’s tree, provided that the given weights 
have been sorted into nondecreasing order: We simply maintain two queues, one 
containing the original weights and the other containing the combined weights. 
At each step the smallest unused weight will appear at the front of one of the 
queues, so we never have to search for it. See exercise 13, which shows that the 
same idea works even when the weights might be negative. 

In general, there are many trees that minimize Yl w jly H the algorithm 
sketched in the preceding paragraph always uses an original weight instead of a 
combined weight in case of ties, then the tree it constructs has the smallest value 
of max^ and of Yhh among all trees that minimize Yl w jlj- H the weights are 
positive, this tree actually minimizes for any convex function /, over 

all such trees. [See E. S. Schwartz, Information and Control 7 (1964). 37-44: 
G. Markowsky, Acta Informatica 16 (1981), 363-370.] 

Huffman’s method can be generalized to t-ary trees as well as binary trees. 
(See exercise 10.) Another important generalization of Huffman’s method is 
discussed in Section 6.2.2. Further discussion of path length appears in Sections 
5.3.1, 5.4.9, and 6.3. 

EXERCISES 

1. [12] Are there any other binary trees with 12 internal nodes and minimum path 
length, besides the complete binary tree ( 5 )? 

2 . [17] Draw an extended binary tree with terminal nodes containing the weights 

I, 4, 9, 16, 25, 36, 49, 64, 81, 100, having minimum weighted path length. 

3. [M24] An extended binary tree with m external nodes determines a set of path 
lengths li,h, ■ ■ ■ ,lm that, describe the lengths of paths from the root to the respective 
external nodes. Conversely, if we are given a set of numbers b , I 2 , • • * , l m , is it always 
possible to construct an extended binary tree in which these numbers are the path 
lengths in some order? Show that this is possible if and only if J2JLi 2~ ,J = 1. 

4. [M25] (E. S. Schwartz and B. Kallick.) Assume that wt < w 2 < • • • < w m . Show 
that there is an extended binary tree that minimizes £ Wjlj and for which the terminal 
nodes in left to right order contain the respective values wi, w 2 , • • • , w m . [For example, 
tree ( 11 ) does not meet this condition since the weights appear in the order 19, 23, 

II, 13, 29, 2, 3, 5, 7, 17, 31, 37, 41. We seek a tree for which the weights appear in 
ascending order, and this does not always happen with Huffman’s construction.] 

5. [ HM26 ] Let 

B(w,z) = ^2 b np w p z n , 
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where b np is the number of binary trees with n nodes and internal path length p. [Thus, 
B(w, z) = 1 + z + 2t vz 2 + ( w 2 + 4 w 3 )z 3 + (4w 4 + 2w s + 8 w 6 )z 4 + ■ ■ • ; 

B(l,z) is the function B(z) of Eq. (13) in Section 2. 3. 4. 4.] 

a) Find a functional relation that characterizes B(w,z), generalizing 2.3.4.4-(i2). 

b) Use the result of (a) to determine the average internal path length of a binary tree 
with n nodes, assuming that each of the ( 2 ”) trees is equally probable. 

c) Find the asymptotic value of this quantity. 

6. [16] If a t-ary tree is extended with square nodes as in (1), what is the relation 
between the number of square and circular nodes corresponding to Eq. (2)? 

7. [ M21 ] What is the relation between external and internal path length in a t-ary 
tree? (See exercise 6; a generalization of Eq. (3) is desired.) 

8. [M23] Prove Eq. (7). 

9. [M21] The numbers that appear in the circular nodes of (11) are equal to the 
sums of the weights in the external nodes of the corresponding subtree. Show that the 
sum of all values in the circular nodes is equal to the weighted path length. 

► 10. [M26] (D. Huffman.) Show how to construct a t-ary tree with minimum weighted 
path length, given nonnegative weights wi,u> 2 , . . . ,w m . Construct an optimal ternary 
tree for weights 1, 4, 9, 16, 25, 36, 49, 64, 81, 100. 

11. [16] Is there any connection between the complete binary tree (5) and the “Dewey 
decimal notation” for binary trees described in exercise 2.3. 1-5? 

► 12. [M20] Suppose that a node has been chosen at random in a binary tree, with each 
node equally likely. Show that the average size of the subtree rooted at that node is 
related to the path length of the tree. 

13. [22] Design an algorithm that begins with m weights w\ < W 2 < • ■ • < w m and 
constructs an extended binary tree having minimum weighted path length. Represent 
the final tree in three arrays 

A[l] . . . A[2m — 1], L[l] . . . L[m — 1], R[l ] . . . R[m - 1]; 

here L[i] and R[i] point to the left and right children of internal node i, the root is 
node 1, and A[i\ is the weight of node i. The original weights should appear as the 
external node weights A[m], . . . , A[2m — 1]. Your algorithm should make fewer than 
2m weight-comparisons. Caution: Some or all of the given weights may be negative! 

14. [25] (T. C. Hu and A. C. Tucker.) After k steps of Huffman’s algorithm, the 
nodes combined so far form a forest of m — k extended binary trees. Prove that this 
forest has the smallest total weighted path length, among all forests of m — k extended 
binary trees that have the given weights. 

15. [M25] Show that a Huffman- like algorithm will find an extended binary tree that 

minimizes (a) max(wi + Zi , . . . ,w m + Z m ); (b) w\x l 1 -I b WmX lm , given x > 1. 

16. [M25] (F. K. Hwang.) Let w\ < ■ ■ ■ < w m and w[ < ■ ■ ■ < w'^ be two sets of 
weights with 

k k 

Y1 w i < 5Z w'j for 1 < k < m. 

3 = 1 3 = 1 

Prove that the minimum weighted path lengths satisfy J2J . li w ih < EJli 
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17. [HM30] (C. R. Glassey and R. M. Karp.) Let si, ..., s m _i be the numbers 
inside the internal (circular) nodes of an extended binary tree formed by Huffman’s 
algorithm, in the order of construction. Let si, . . . , s' m ^ 1 be the internal node weights 
of any extended binary tree on the same set of weights {wi, . . . ,w m }, listed in any 
order such that each nonroot internal node appears before its parent, (a) Prove that 
J2j= i s i £ X!j= i s 'j f° r 1 < k < m. (b) The result of (a) is equivalent to 

m — 1 m— 1 

£ /(«,-) < £ /(«;■) 

j=i i = i 

for every nondecreasing concave function /, namely every function / with /'( x) > 0 
and f"(x) < 0. [See Hardy, Littlewood, and Polya, Messenger of Math. 58 (1929), 
145-152.] Use this fact to show that the minimum value in the recurrence 

F( n ) = f(n)+ min (F(k) + F(n — k)) , F( 1) = 0 

1 <k<n 

always occurs when k = 2 ris(n/3)1 , given any function f(n) with the property that 
A/(n) = f(n + 1) - f(n) > 0 and A 2 /(n) = A f(n + 1) - A f(n) < 0. 

*2. 3. 4. 6 . History and bibliography. Trees have of course been in existence 
since the third day of creation, and through the ages tree structures (especially 
family trees) have been in common use. The concept of tree as a formally defined 
mathematical entity seems to have appeared first in the work of G. Kirchhoff 
[Annaien der Physik and Chemie 72 (1847), 497-508, English translation in 
IRE Transactions CT-5 (1958), 4-7]; Kirchhoff used free trees to find a set of 
fundamental cycles in an electrical network in connection with the law that bears 
his name, essentially as we did in Section 2.3.4. 1 . The concept also appeared at 
about the same time in the book Geometrie der Lage (pages 20-21) by K. G. Chr. 
von Staudt. The name “tree” and many results dealing mostly with enumeration 
of trees began to appear ten years later in a series of papers by Arthur Cayley [see 
Collected Mathematical Papers of A. Cayley 3 (1857), 242-246; 4 (1859), 112— 
115; 9 (1874), 202-204; 9 (1875), 427-460; 10 (1877), 598-600; 11 (1881), 365- 
367; 13 (1889), 26-28]. Cayley was unaware of the previous work of Kirchhoff 
and von Staudt; his investigations began with studies of the structure of algebraic 
formulas, and they were later inspired chiefly by applications to the problem of 
isomers in chemistry. Tree structures were also studied independently by C. W. 
Borchardt [Crelle 57 (1860), 111-121]; J. B. Listing [Gottinger Abhandlungen, 
Math. Classe, 10 (1862), 137-139]; and C. Jordan [Crelle 70 (1869), 185-190], 
The “infinity lemma” was formulated first by Denes Konig [Fundamenta 
Math. 8 (1926), 114-134], and he gave it a prominent place in his classic book 
Theorie der endlichen und unendlichen Graphen (Leipzig: 1936), Chapter 6 . 
A similar result called the “fan theorem” occurred slightly earlier in the work 
of L. E. J. Brouwer [Verhandelingen Akad. Amsterdam 12 (1919), 7], but this 
involved much stronger hypotheses; see A. Heyting, Intuitionism (1956), Sec- 
tion 3.4, for a discussion of Brouwer’s work. 

Formula ( 3 ) of Section 2. 3. 4. 4 for enumerating unlabeled oriented trees was 
given by Cayley in his first paper on trees. In his second paper he enumerated 
unlabeled ordered trees; an equivalent problem in geometry (see exercise 1 ) 
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had already been proposed and solved by L. Euler, who mentioned his results 
in a letter to C. Goldbach on 4 September 1751 [see J. von Segner and L. 
Euler, Novi Commentarii Academias Scientiarum Petropolitanas 7 (1758-1759), 
summary 13-15, 203-210], Euler’s problem was the subject of seven papers by 
G. Lame, E. Catalan, O. Rodrigues, and J. Binet in Journal de mathematiques 
3, 4 (1838, 1839); additional references appear in the answer to exercise 2.2. 1-4. 
The corresponding numbers are now commonly called “Catalan numbers.” A 
Mongolian Chinese mathematician, An-T’u Ming, had encountered the Catalan 
numbers before 1750 in his study of infinite series, but he did not relate them to 
trees or other combinatorial objects [see J. Luo, Acta Scientiarum Naturalium 
Universitatis Intramongolicse 19 (1988), 239-245; Combinatorics and Graph 
Theory (World Scientific Publishing, 1993), 68-70], Catalan numbers occur in 
an enormous number of different contexts; Richard Stanley explains more than 
60 of them in his magnificent book Enumerative Combinatorics 2 (Cambridge 
Univ. Press, 1999), Chapter 6. Perhaps most surprising of all is the Catalan 
connection to certain arrangements of numbers that H. S. M. Coxeter has called 
“frieze patterns” because of their symmetry; see exercise 4. 

The formula n" -2 for the number of labeled free trees was discovered by J. J. 
Sylvester [Quart. J. Pure and Applied Math. 1 (1857), 55-56], as a byproduct 
of his evaluation of a certain determinant (exercise 2.3.4.2-28). Cayley gave 
an independent derivation of the formula in 1889 [see the reference above]; 
his discussion, which was extremely vague, hinted at a connection between 
labeled oriented trees and (n — l)-tuples of numbers. An explicit correspondence 
demonstrating such a connection was first published by Heinz Priifer [Arch. 
Math. undPhys. 27 (1918), 142-144], quite independently of Cayley’s prior work. 
A large literature on this subject has developed, and the classical results are 
surveyed beautifully in J. W. Moon’s book, Counting Labelled Trees (Montreal: 
Canadian Math. Congress, 1970). 

A very important paper on the enumeration of trees and many other kinds 
of combinatorial structures was published by G. Polya in Acta Math. 68 (1937), 
145-253. For a discussion of enumeration problems for graphs and an excellent 
bibliography of the early literature, see the survey by Frank Harary in Graph 
Theory and Theoretical Physics (London: Academic Press, 1967), 1-41. 

The principle of minimizing weighted path length by repeatedly combining 
the smallest weights was discovered by D. Huffman [ Proc . IRE 40 (1952), 1098- 
1101], in connection with the design of codes for minimizing message lengths. 
The same idea was independently published by Seth Zimmerman [AMM 66 
(1959), 690-693], 

Several other noteworthy papers about the theory of tree structures have 
been cited in Sections 2.3.4. 1 through 2. 3. 4. 5 in connection with particular topics. 

EXERCISES 

► 1. [21] Find a simple one-to-one correspondence between binary trees with n nodes 

and dissections of an (n + 2)-sided convex polygon into n triangles, assuming that the 
sides of the polygon are distinct. 
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► 2. [M26] T . P. Kirkman conjectured in 1857 that the number of ways to draw A: non- 
overlapping diagonals in an r-sided polygon is ( iXT) ( r k 3 ) / ( r + k )• 

a) Extend the correspondence of exercise 1 to obtain an equivalent problem about 
the enumeration of trees. 

b) Prove Kirkman’s conjecture by using the methods of exercise 2.3.4.4-32. 

► 3. [M30] Consider all ways of partitioning the vertices of a convex n- gon into k non- 
empty parts, in such a way that no diagonal between two vertices of one part crosses 
a diagonal between two vertices of another part. 

a) Find a one-to-one correspondence between noncrossing partitions and an interest- 
ing class of tree structures. 

b) Given n and k, how many ways are there to make such a partition? 

► 4. [M38] (Conway and Coxeter.) A frieze pattern is an infinite array such as 

1111111111111111111 
3131412313141231314 ... 

522233 1 522233152223 
3315222331522233152 ... 

1412313141231314123 
1111111111111111111 

in which the top and bottom rows consist entirely of Is, and each diamond of adjacent 
values a c d satisfies ad — be = 1 . Find a one-to-one correspondence between n-node 
binary trees and ( n + l)-rowed frieze patterns of positive integers. 

2.3.5. Lists and Garbage Collection 

Near the beginning of Section 2.3 we defined a List informally as “a finite 
sequence of zero or more atoms or Lists.” 

Any forest is a List; for example, 


a e 



h 


may be regarded as the List 

(a: ( b , c, d), e: (/,g: ( h ))), 
and the corresponding List diagram would be 


* 



h 


The reader should review at this point the introduction to Lists given earlier, 
in particular ( 3 ), ( 4 ), ( 5 ), ( 6 ), ( 7 ) in the opening pages of Section 2.3. Recall that, 
in ( 2 ) above, the notation “a: ( 6 , c, d)' means that (h, c, d) is a List of three atoms, 
which has been labeled with the attribute “a” . This convention is compatible 


2.3.5 


LISTS AND GARBAGE COLLECTION 409 


with our general policy that each node of a tree may contain information besides 
its structural connections. However, as was discussed for trees in Section 2.3.3, 
it is quite possible and sometimes desirable to insist that all Lists be unlabeled, 
so that all the information appears in the atoms. 

Although any forest may be regarded as a List, the converse is not true. 
The following List is perhaps more typical than ( 2 ) and ( 3 ) since it shows how 
the restrictions of tree structure might be violated: 

L= (a:N,b,c:(d:N),e:L), N = (/: (),g: (h: L,j: N)) ( 4 ) 

which may be diagrammed as 



[LI 


a*[iV] b c * e[L\ 
/* g* d[N } 


h[L] i[iV] 


(5) 


[Compare with the example in 2.3 ( 7 ). The form of these diagrams need not be 
taken too seriously.] 

As we might expect, there are many ways to represent List structures within 
a computer memory. These methods are usually variations on the same basic 
theme by which we have used binary trees to represent general forests of trees: 
One field, say RLINK, is used to point to the next element of a List, and another 
field DLINK may be used to point to the first element of a sub-List. By a natural 
extension of the memory representation described in Section 2.3.2, we would 
represent the List ( 5 ) as follows: 



( 6 ) 


Unfortunately, this simple idea is not quite adequate for the most common 
List processing applications. For example, suppose that we have the List L = 
[A, a, (A, A)) , which contains three references to another List A = ( 6 , c, d). One 
of the typical List processing operations is to remove the leftmost element of A, 
so that A becomes (c, d); but this requires three changes to the representation 
of L , if we are to use the technique shown in ( 6 ), since each pointer to A points 
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to the element b that is being deleted. A moment’s reflection will convince the 
reader that it is extremely undesirable to change the pointers m every reference 
to A just because the first element of A is being deleted. (In this example we 
could try to be tricky, assuming that there are no pointers to the element c, by 
copying the entire element c into the location formerly occupied by b and then 
deleting the old element c. But this trick fails to work when A loses its last 
element and becomes empty.) 

For this reason the representation scheme ( 6 ) is generally replaced by another 
scheme that is similar, but uses a List head to begin each List, as was introduced 
in Section 2.2.4. Each List contains an additional node called its List head, so 
that the configuration ( 6 ) would, for example, be represented thus: 


(7) 


The introduction of such header nodes is not really a waste of memory space 
in practice, since many uses for the apparently unused fields — the shaded areas 
in diagram ( 7 ) generally present themselves. For example, there is room for a 
reference count, or a pointer to the right end of the List, or an alphabetic name, 
or a “scratch’ field that aids traversal algorithms, etc. 

In our original diagram ( 6 ), the node containing b is an atom while the node 
containing / specifies an empty List. These two things are structurally identical, 
so the reader would be quite justified in asking why we bother to talk about 
“atoms” at all; with no loss of generality we could have defined Lists as merely 
“a finite sequence of zero or more Lists,” with our usual convention that each 
node of a List may contain data besides its structural information. This point 
of view is certainly defensible and it makes the concept of an “atom” seem very 
artificial. There is, however, a good reason for singling out atoms as we have 
done, when efficient use of computer memory is taken into consideration, since 
atoms are not subject to the same sort of general-purpose manipulation that is 
desired for Lists. The memory representation ( 6 ) shows there is probably more 
room for information in an atomic node, 6 , than in a List node, /; and when 
List head nodes are also present as in ( 7 ), there is a dramatic difference between 
the storage requirements for the nodes b and /. Thus the concept of atoms is 
introduced primarily to aid in the effective use of computer memory. Typical 
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Lists contain many more atoms than our example would indicate; the example of 
UM7) is intended to show the complexities that are possible, not the simplicities 
that are usual. 

A List is in essence nothing more than a linear list whose elements may 
contain pointers to other Lists. The common operations we wish to perform on 
Lists are the usual ones desired for linear lists (creation, destruction, insertion, 
deletion, splitting, concatenation), plus further operations that are primarily of 
interest for tree structures (copying, traversal, input and output of nested infor- 
mation). For these purposes any of the three basic techniques for representing 
linked linear lists in memory — namely straight, circular, or double linkage — can 
be used, with varying degrees of efficiency depending on the algorithms being 
employed. For these three types of representation, diagram ( 7 ) might appear in 
memory as follows: 


Memory Straight linkage 
location INFO DLINK ELINK 

Circular linkage 

INFO DLINK RLINK 

INFO 

Double linkage 

DLINK LLINK 

RLINK 

010 


head 

020 

— 

head 

020 

— 

head 

050 

020 

020 

a 

060 

030 

a 

060 

030 

a 

060 

010 

030 

030 

b 

atom 

040 

b 

atom 

040 

b 

atom 

020 

040 

040 

c 

090 

050 

c 

090 

050 

c 

090 

030 

050 

050 

e 

010 

A 

e 

010 

010 

e 

010 

040 

010 

060 


head 

070 

— 

head 

070 

— 

head 

080 

070 

070 

f 

110 

080 

/ 

110 

080 

/ 

110 

060 

080 

080 

9 

120 

A 

9 

120 

060 

9 

120 

070 

060 

090 


head 

100 

— 

head 

100 

— 

head 

100 

100 

100 

d 

060 

A 

d 

060 

090 

d 

060 

090 

090 

110 


head 

A 

— 

head 

110 

— 

head 

110 

no 

120 


head 

130 

— 

head 

130 

— 

head 

140 

130 

130 

h 

010 

140 

h 

010 

140 

h 

010 

120 

140 

140 

j 

060 

A 

j 

060 

120 

j 

060 

130 

120 


Here "LLINK is used for a pointer to the left in a doubly linked representation. 
The INFO and DLINK fields are identical in all three forms. 

There is no need to repeat here the algorithms for List manipulation in any 
of these three forms, since we have already discussed the ideas many times. The 
following important points about Lists, which distinguish them from the simpler 
special cases treated earlier, should however be noted: 

1) It is implicit in the memory representation above that atomic nodes are 
distinguishable from nonatomic nodes; furthermore, when circular or doubly 
linked Lists are being used, it is desirable to distinguish header nodes from the 
other types, as an aid in traversing the Lists. Therefore each node generally 
contains a TYPE field that tells what kind of information the node represents. 
This TYPE field is often used also to distinguish between various types of atoms 
(for example, between alphabetic, integer, or floating point quantities, for use 
when manipulating or displaying the data) . 

2) The format of nodes for general List manipulation with the MIX computer 
might be designed in one of the following two ways. 
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a) Possible one-word format, assuming that all INFO appears in atoms: 


s 

T 

REF 

RLINK 


(9) 


S (sign): 
T (type): 

REF: 


RLINK: 


Mark bit used in garbage collection (see below). 

T = 0 for List head; T = 1 for sub-List element; T > 1 for atoms. 
When T = 0, REF is a reference count (see below); when T = 1. 
REF points to the List head of the sub-List in question: when 
T > 1, REF points to a node containing a mark bit and five bytes 
of atomic information. 

Pointer for straight or circular linkage as in (8). 


b) Possible two- word format: 


S 


LLINK 

RLINK 



INFO 



As in (9). 

The usual pointers for double linkage as in (8). 

A full word of information associated with this node; for a 
header node this may include a reference count, a running 
pointer to the interior of the List to facilitate linear traversal, 
an alphabetic name, etc. When T = 1 this information 
includes DLINK. 

3) It is clear that Lists are very general structures; indeed, it seems fair to 
state that any structure whatsoever can be represented as a List when appropri- 
ate conventions are made. Because of this universality of Lists, a large number 
of programming systems have been designed to facilitate List manipulation, and 
there are usually several such systems available at any computer installation. 
Such systems are based on a general-purpose format for nodes such as (9) or 
(10) above, designed for flexibility in List operations. Actually, it is clear that 
this general-purpose format is usually not the best format suited to a partic- 
ular application, and the processing time using the general-purpose routines 
is noticeably slower than a person would achieve by hand-tailoring the system 
to a particular problem. For example, it is easy to see that nearly all of the 
applications we have worked out so far in this chapter would be encumbered by 
a general-List representation as in (9) or (10) instead of the node format that was 
given in each case. A List manipulation routine must often examine the T-field 
when it processes nodes, and that was not needed in any of our programs so 
far. This loss of efficiency is repaid in many instances by the comparative ease 
of programming and the reduction of debugging time when a general-purpose 
system is used. 

4) There is also an extremely significant difference between algorithms for 
List processing and the algorithms given previously in this chapter. Since a single 


S, T: 

LLINK, RLINK: 
INFO: 
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List may be contained in many other Lists, it is by no means clear exactly when 
a List should be returned to the pool of available storage. Our algorithms so far 
have always said "AVAIL <= X”, whenever NODE(X) was no longer needed. But 
since general Lists can grow and die in a completely unpredictable manner, it is 
often quite difficult to tell just when a particular node is superfluous. Therefore 
the problem of maintaining the list of available space is considerably more 
difficult with Lists than it was in the simple cases considered previously. We will 
devote the rest of this section to a discussion of the storage reclamation problem. 

Let us imagine that we are designing a general-purpose List processing 
system that will be used by hundreds of other programmers. Two principal 
methods have been suggested for maintaining the available space list: the use 
of reference counters , and garbage collection. The reference-counter technique 
makes use of a new field in each node, which contains a count of how many 
arrows point to this node. Such a count is rather easy to maintain as a program 
runs, and whenever it drops to zero, the node in question becomes available. 
The garbage-collection technique, on the other hand, requires a new one-bit field 
in each node called the mark bit. The idea in this case is to write nearly all 
the algorithms so that they do not return any nodes to free storage, and to let 
the program run merrily along until all of the available storage is gone; then a 
“recycling” algorithm makes use of the mark bits to identify all nodes that are 
not currently accessible and to return them to available storage, after which the 
program can continue. 

Neither of these two methods is completely satisfactory. The principal 
drawback of the reference-counter method is that it does not always free all 
the nodes that are available. It works fine for overlapped Lists (Lists that 
contain common sub-Lists); but recursive Lists, like our examples L and N in 
(4), will never be returned to storage by the reference-counter technique. Their 
counts will be nonzero (since they refer to themselves) even when no other List 
accessible to the running program points to them. Furthermore, the reference- 
counter method uses a good chunk of space in each node (although this space is 
sometimes available anyway due to the computer word size). 

The difficulty with the garbage-collection technique, besides the annoying 
loss of a bit in each node, is that it runs very slowly when nearly all the memory 
space is in use; and in such cases the number of free storage cells found by the 
reclamation process is not worth the effort. Programs that exceed the capacity 
of storage (and many undebugged programs do!) often waste a good deal of time 
calling the garbage collector several almost fruitless times just before storage is 
finally exhausted. A partial solution to this problem is to let the programmer 
specify a number fc, signifying that processing should not continue after a garbage 
collection run has found k or fewer free nodes. 

Another problem is the occasional difficulty of determining exactly what 
Lists are not garbage at a given stage. If the programmer has been using any 
nonstandard techniques or keeping any pointer values in unusual places, chances 
are good that the garbage collector will go awry. Some of the greatest mysteries 
in the history of debugging have been caused by the fact that garbage collection 
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suddenly took place at an unexpected time during the running of programs 
that had worked many times before. Garbage collection also requires that 
programmers keep valid information in all pointer fields at all times, although 
we often find it convenient to leave meaningless information in fields that the 
piogram doesn t use for example, the link in the rear node of a queue; see 
exercise 2. 2. 3-6. 

Although garbage collection requires one mark bit for each node, we could 
keep a separate table of all the mark bits packed together in another memory 
area, with a suitable correspondence between the location of a node and its mark 
bit. On some computers this idea can lead to a method of handling garbage 
collection that is more attractive than giving up a bit in each node. 

J. Weizenbaum has suggested an interesting modification of the reference- 
counter technique. Using doubly linked List structures, he puts a reference 
counter only in the header of each List. Thus, when pointer variables traverse 
a List, they are not included in the reference counts for the individual nodes. If 
we know the rules by which reference counts are maintained for entire Lists, we 
know (in theory) how to avoid referring to any List that has a reference count 
of zero. We also have complete freedom to explicitly override reference counts 
and to return particular Lists to available storage. These ideas require careful 
handling; they prove to be somewhat dangerous in the hands of inexperienced 
programmers, and they’ve tended to make program debugging more difficult due 
to the consequences of referring to nodes that have been erased. The nicest part 
of Weizenbaum s approach is his treatment of Lists whose reference count has just 
gone to zero: Such a List is appended at the end of the current available list — 
this is easy to do with doubly linked Lists — and it is considered for available 
space only after all previously available cells are used up. Eventually, as the 
individual nodes of this List do become available, the reference counters of Lists 
they refer to are decreased by one. This delayed action of erasing Lists is quite 
efficient with respect to running time; but it tends to make incorrect programs 
run correctly for awhile! For further details see CACM 6 (1963), 524-544. 

Algorithms for garbage collection are quite interesting for several reasons. 
In the first place, such algorithms are useful in other situations when we want to 
mark all nodes that are directly or indirectly referred to by a given node. (For 
example, we might want to find all subroutines called directly or indirectly by a 
certain subroutine, as in exercise 2.2.3 26.) 

Garbage collection generally proceeds in two phases. We assume that the 
mark bits of all nodes are initially zero (or we set them all to zero). Now the first 
phase marks all the nongarbage nodes, starting from those that are immediately 
accessible to the main program. The second phase makes a sequential pass over 
the entire memory pool area, putting all unmarked nodes onto the list of free 
space. The marking phase is the most interesting, so we will concentrate our 
attention on it. Certain variations on the second phase can, however, make it 
nontrivial; see exercise 9. 

When a garbage collection algorithm is running, only a very limited amount 
of stoiage is available to control the marking procedure. This intriguing problem 
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will become clear in the following discussion; it is a difficulty that is not appre- 
ciated by most people when they first hear about the idea of garbage collection, 
and for several years there was no good solution to it. 

The following marking algorithm is perhaps the most obvious. 

Algorithm A (Marking). Let the entire memory used for List storage be 
N0DE(1) , N0DE(2) , . . . , NODE(M) , and suppose that these words either are atoms 
or contain two link fields ALINK and BLINK. Assume that all nodes are initially 
unmarked. The purpose of this algorithm is to mark all of the nodes that can be 
reached by a chain of ALINK and/or BLINK pointers in nonatomic nodes, starting 
from a set of “immediately accessible” nodes, that is, nodes pointed to by certain 
fixed locations in the main program; these fixed pointers are used as a source for 
all memory accesses. 

Al. [Initialize.] Mark all nodes that are immediately accessible. Set K t— 1. 

A2. [Does NODE(K) imply another?] Set K1 <S— K + 1. If NODE(K) is an atom 
or unmarked, go to step A3. Otherwise, if NODE (ALINK (K) ) is unmarked: 
Mark it and, if it is not an atom, set K1 -t— min(Kl, ALINK(K) ). Similarly, 
if NODE (BLINK (K) ) is unmarked: Mark it and, if it is not an atom, set 
K1 <- min(Kl, BLINK(K) ). 

A3. [Done?] Set K •<— Kl. If K < M, return to step A2; otherwise the algorithm 
terminates. | 

Throughout this algorithm and the ones that follow in this section, we will 
assume for convenience that the nonexistent node “NODE (A)” is marked. (For 
example, ALINK(K) or BLINK(K) may equal A in step A2.) 

A variant of Algorithm A sets Kl <— M+ 1 in step Al, removes the operation 
“Kl «— K + 1” from step A2, and instead changes step A3 to 

A3'. [Done?] Set K t— K + 1. If K < M, return to step A2. Otherwise if Kl < M, 
set K <— Kl and Kl f- M + 1 and return to step A2. Otherwise the algorithm 
terminates. 

It is very difficult to give a precise analysis of Algorithm A, or to determine 
whether it is better or worse than the variant just described, since no meaningful 
way to describe the probability distribution of the input presents itself. We can 
say that it takes up time proportional to nM in the worst case, where n is the 
number of cells it marks; and, in general, we can be sure that it is very slow 
when n is large. Algorithm A is too slow to make garbage collection a usable 
technique. 

Another fairly evident marking algorithm is to follow all paths and to record 
branch points on a stack as we go: 

Algorithm B (Marking). This algorithm achieves the same effect as Algo- 
rithm A, using STACK [1] , STACK [2] , . . . as auxiliary storage to keep track of all 
paths that have not yet been pursued to completion. 

Bl. [Initialize.] Let T be the number of immediately accessible nodes; mark 
them and place pointers to them in STACK [1] , . . . , STACK [T] . 
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B2. [Stack empty?] If T = 0, the algorithm terminates. 

B3. [Remove top entry.] Set K «— STACK [T] , T t— T — 1. 

B4. [Examine links.] If NODE(K) is an atom, return to step B2. Otherwise, if 
N0DE(ALINK(K) ) is unmarked, mark it and set T «- T + 1, STACK [T] «- 
ALINK(K) ; if NODE (BLINK (K) ) is unmarked, mark it and set T T + 1. 
STACK [T] f- BLINK (K). Return to B2. | 

Algorithm B clearly has an execution time essentially proportional to the 
number of cells it marks, and this is as good as we could possibly expect; but it 
is not really usable for garbage collection because there is no place to keep the 
stack! It does not seem unreasonable to assume that the stack in Algorithm B 
might grow up to, say, five percent of the size of memory; but when garbage 
collection is called, and all available space has been used up, there is only a 
fixed (rather small) number of cells to use for such a stack. Most of the early 
garbage collection procedures were essentially based on this algorithm. If the 
special stack space was used up, the entire program had to be terminated. 

A somewhat better alternative is possible, using a fixed stack size, by com- 
bining Algorithms A and B: 

Algorithm C (Marking). This algorithm achieves the same effect as Algo- 
rithms A and B, using an auxiliary table of H cells, STACK [0], STACK [1] 

STACK [H - 1] . 

In this algorithm, the action “insert X on the stack” means the following: 
Set T 4 (T + 1) mod H, and STACK [T] < — X. If T = B, set B < — (B + 1) mod H and 
K1 min(Kl, STACK [B] ).” (Note that T points to the current top of the stack, 
and B points one place below the current bottom; STACK essentially operates as 
an input-restricted deque.) 

Cl. [Initialize.] Set T f- H — 1, B f- H — 1, Kle-M+1. Mark all the immediately 
accessible nodes, and successively insert their locations onto the stack (as 
just described above). 

C2. [Stack empty?] If T = B, go to C5. 

C3. [Remove top entry.] Set K STACK [T] , T <- (T — 1) mod H. 

C4. [Examine links.] If NODE(K) is an atom, return to step C2. Otherwise, if 
N0DE(ALINK(K) ) is unmarked, mark it and insert ALINK(K) on the stack. 
Similarly, if N0DE(BLINK(K) ) is unmarked, mark it and insert BLINK (K) on 
the stack. Return to C2. 

C5. [Sweep.] If K1 > M, the algorithm terminates. (The variable K1 represents 
the smallest location where there is a possibility of a new lead to a node 
that should be marked.) Otherwise, if N0DE(K1) is an atom or unmarked, 
increase K1 by 1 and repeat this step. If N0DE(K1) is marked, set K <— Kl. 
increase Kl by 1, and go to C4. | 

This algorithm and Algorithm B can be improved if X is never put on the 
stack when NODE(X) is an atom; moreover, steps B4 and C4 need not put items 
on the stack when they know that the items will immediately be removed. Such 
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modifications are straightforward and they have been left out to avoid making 
the algorithms unnecessarily complicated. 

Algorithm C is essentially equivalent to Algorithm A when H = 1, and to 
Algorithm B when H = M; it gradually becomes more efficient as H becomes 
larger. Unfortunately, Algorithm C defies a precise analysis for the same reason 
as Algorithm A, and we have no good idea how large H should be to make this 
method fast enough. It is plausible but uncomfortable to say that a value like 
H = 50 is sufficient to make Algorithm C usable for garbage collection in most 
applications. 

Algorithms B and C use a stack kept in sequential memory locations; but we 
have seen earlier in this chapter that linked memory techniques are well suited to 
maintaining stacks that are not consecutive in memory. This suggests the idea 
that we might keep the stack of Algorithm B somehow scattered through the 
same memory area in which we are collecting garbage. This could be done easily 
if we were to give the garbage collection routine a little more room in which 
to breathe. Suppose, for example, we assume that all Lists are represented as 
in (9)1 except that the REF fields of List head nodes are used for garbage collection 
purposes instead of as reference counts. We can then redesign Algorithm B so 
that the stack is maintained in the REF fields of the header nodes: 

Algorithm D (Marking). This algorithm achieves the same effect as Algorithms 
A, B, and C, but it assumes that the nodes have S, T, REF. and RLINK fields as 
described above, instead of ALINKs and BLINKs. The S field is used as the mark 
bit, so that S (P) == 1 means that NODE(P) is marked. 

Dl. [Initialize.] Set TOP 4— A. Then for each pointer P to the head of an 
immediately accessible List (see step A1 of Algorithm A), if S(P) = 0, set 
S(P) 4 - 1 , REF (P) 4- TOP, TOP 4 - P. 

D2. [Stack empty?] If TOP = A, the algorithm terminates. 

D3. [Remove top entry.] Set P 4- TOP. TOP 4- REF(P) . 

D4. [Move through List.] Set P 4- RLINK (P) ; then if P = A, or if T(P) = 0, go 
to D2. Otherwise set S(P) 4— 1. If T(P) > 1, set S(REF(P)) 4 — 1 (thereby 
marking the atomic information). Otherwise (T(P) = 1), set Q 4 - REF(P) ; 
if Q / A and S (Q) = 0, set S(Q) 4 - 1, REF (Q) 4- TOP, TOP 4- Q. Repeat 
step D4. | 

Algorithm D may be compared to Algorithm B, which is quite similar, and 
its running time is essentially proportional to the number of nodes marked. 
However, Algorithm D is not recommended without qualification, because its 
seemingly rather mild restrictions are often too stringent for a general List- 
processing system. This algorithm essentially requires that all List structures 
be well-formed, as in ( 7 ), whenever garbage collection is called into action. 
But algorithms for List manipulations momentarily leave the List structures 
malformed, and a garbage collector such as Algorithm D must not be used during 
those momentary periods. Moreover, care must be taken in step Dl when the 
program contains pointers to the middle of a List. 
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After After 

ALINK BLINK 



Fig. 38. Algorithm E for marking with no auxiliary stack space. 


These considerations bring us to Algorithm E, which is an elegant marking 
method discovered independently by Peter Deutsch and by Herbert Schorr and 
W. M. Waite in 1965. The assumptions used in this algorithm are just a little 
different from those of Algorithms A through D. 

Algorithm E (Marking). Assume that a collection of nodes is given having the 
following fields: 

MARK (a one-bit field), 

ATOM (another one-bit field), 

ALINK (a pointer field), 

BLINK (a pointer field). 

When ATOM = 0, the ALINK and BLINK fields may contain A or a pointer to 
another node of the same format; when ATOM = 1, the contents of the ALINK and 
BLINK fields are irrelevant to this algorithm. 

Given a nonnull pointer P0, this algorithm sets the MARK field equal to 1 in 
NODE (P0) and in every other node that can be reached from NODE(PO) by a chain 
of ALINK and BLINK pointers in nodes with ATOM = MARK = 0. The algorithm 
uses three pointer variables, T, Q, and P. It modifies the links and control bits in 
such a way that all ATOM, ALINK, and BLINK fields are restored to their original 
settings after completion, although they may be changed temporarily. 

El. [Initialize.] Set T f- A, P f- P0. (Throughout the remainder of this 
algorithm, the variable T has a dual significance: When T ^ A, it points 
to the top of what is essentially a stack as in Algorithm D; and the node 
that T points to once contained a link equal to P in place of the “artificial” 
stack link that currently occupies NODE(T).) 

E2. [Mark.] Set MARK(P) 4- 1. 

E3. [Atom?] If ATOM(P) = 1, go to E6. 

E4. [Down ALINK.] Set Q 4- ALINK (P) . If Q / A and MARK(Q) = 0, set 
ATOM(P) 4— 1, ALINK(P) 4— T, T 4— P, P 4— Q, and go to E2. (Here the 
ATOM field and ALINK fields are temporarily being altered, so that the List 
structure in certain marked nodes has been rather drastically changed. But 
these changes will be restored in step E6.) 

E5. [Down BLINK.] Set Q 4- BLINK (P). If Q / A and MARK(Q) = 0, set 
BLINK (P) 4— T, T 4— P, P 4— Q, and go to E2. 
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ALINK [MARK] b[ 0] 
BLINK [ATOM] c[0] 
ALINK [MARK] -[0] 
BLINK [ATOM] -[1] 
ALINK [MARK] b[ 0] 
BLINK [ATOM] d[ 0] 
ALINK [MARK] e[0] 
BLINK [ATOM] d[ 0] 
ALINK [MARK] A[0] 
BLINK [ATOM] c[0] 



T — AaaAaacdddccaA 
P — abbaccdeeeddc a 


Next step El E2 E2 E6 E5 E2 E5 E2 E2 E5 E6 E5 E6 E6 E6 

I 1 1 III! 1 1 1 | I | | 

Nesting I I L 1 — 'I 


Fig. 39. A structure marked by Algorithm E. (The table shows only changes that 
have occurred since the previous step.) 


E6. [Up.] (This step undoes the link switching made in step E4 or E5; the 
setting of AT0M(T) tells whether ALINK (T) or BLINK (T) is to be restored.) 
If T = A, the algorithm terminates. Otherwise set Q T. If AT0M(Q) = 1, 
set AT0M(Q) 0, T ALINK(Q), ALINK(Q) P, P <— Q, and return to E5. 
If AT0M(Q) = 0, set T <— BLINK (Q) , BLINK (Q) <- P, P <- Q, and repeat E6. | 

An example of this algorithm in action appears in Fig. 39, which shows the 
successive steps encountered for a simple List structure. The reader will find it 
worthwhile to study Algorithm E very carefully; notice how the linking structure 
is artificially changed in steps E4 and E5, in order to maintain a stack analogous 
to the stack in Algorithm D. When we return to a previous state, the ATOM 
field is used to tell whether ALINK or BLINK contains the artificial address. The 
“nesting’’ shown at the bottom of Fig. 39 illustrates how each nonatomic node 
is visited three times during Algorithm E: The same configuration (T,P) occurs 
at the beginning of steps E2, E5, and E6. 
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A proof that Algorithm E is valid can be formulated by induction on the 
number of nodes that are to be marked. We prove at the same time that P 
returns to its initial value P0 at the conclusion of the algorithm; for details see 
exercise 3. Algorithm E will run faster if step E3 is deleted and if special tests 
for “ATOM(Q) = 1” and appropriate actions are made in steps E4 and E5, as 
well as a test “ATOM(PO) = 1” in step El. We have stated the algorithm in its 
present form for simplicity; the modifications just stated appear in the answer 
to exercise 4. 

The idea used in Algorithm E can be applied to problems other than garbage 
collection; in fact, its use for tree traversal has already been mentioned in 
exercise 2.3.1-21. The reader may also find it useful to compare Algorithm E 
with the simpler problem solved in exercise 2 . 2 . 3 - 7 . 

Of all the marking algorithms we have discussed, only Algorithm D is 
directly applicable to Lists represented as in ( 9 ). The other algorithms all 
test whether or not a given node P is an atom, and the conventions of ( 9 ) are 
incompatible with such tests because they allow atomic information to fill an 
entire word except for the mark bit. However, each of the other algorithms 
can be modified so that they will work when atomic data is distinguished from 
pointer data in the word that links to it instead of by looking at the word 
itself. In Algorithms A or C we can simply avoid marking atomic words until 
all nonatomic words have been properly marked; then one further pass over all 
the data suffices to mark all the atomic words. Algorithm B is even easier to 
modify, since we need merely keep atomic words off the stack. The adaptation of 
Algorithm E is almost as simple, although if both ALINK and BLINK are allowed 
to point to atomic data it will be necessary to introduce another 1 -bit. field in 
nonatomic nodes. This is generally not hard to do. (For example, when there 
are two words per node, the least significant bit of each link field may be used 
to store temporary information.) 

Although Algorithm E requires a time proportional to the number of nodes 
it marks, this constant of proportionality is not as small as in Algorithm B: 
the fastest garbage collection method known combines Algorithms B and E. as 
discussed in exercise 5 . 

Let us now try to make some quantitative estimates of the efficiency of 
garbage collection, as opposed to the philosophy of “AVAIL <t= X” that was 
used in most of the previous examples in this chapter. In each of the previous 
cases we could have omitted all specific mention of returning nodes to free 
space and we could have substituted a garbage collector instead. (I11 a special- 
purpose application, as opposed to a set of general-purpose List manipulation 
subroutines, the programming and debugging of a garbage collector is more 
difficult than the methods we have used, and, of course, garbage collection 
requires an extra bit reserved in each node; but we are interested here in the 
relative speed of the programs once they have been written and debugged.) 

The best garbage collection routines known have an execution time essen- 
tially of the form c x N + c 2 M, where Cl and c 2 are constants, N is the number of 
nodes marked, and M is the total number of nodes in the memory. Thus M - N is 
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the number of free nodes found, and the amount of time required to return these 
nodes to free storage is (cjN + c 2 M)/(M - N) per node. Let N = pM; this figure 
becomes (cip+c 2 )/ (1 — p). So if p = that is, if the memory is three-fourths full, 
we spend 3ci + 4c 2 units of time per free node returned to storage; when p = |, 
the corresponding cost is only + |c 2 . If we do not use the garbage collection 
technique, the amount of time per node returned is essentially a constant, C 3 , 
and it is doubtful that c 3 /ci will be very large. Hence we can see to what extent 
garbage collection is inefficient when the memory becomes full, and how it is 
correspondingly efficient when the demand on memory is light. 

Many programs have the property that the ratio p = N/M of good nodes 
to total memory is quite small. When the pool of memory becomes full in such 
cases, it might be best to move all the active List data to another memory pool of 
equal size, using a copying technique (see exercise 10 ) but without bothering to 
preserve the contents of the nodes being copied. Then when the second memory 
pool fills up, we can move the data back to the first one again. With this method 
more data can be kept in high-speed memory at once, because link fields tend 
to point to nearby nodes. Moreover, there’s no need for a marking phase, and 
storage allocation is simply sequential. 

It is possible to combine garbage collection with some of the other methods 
of returning cells to free storage; these ideas are not mutually exclusive, and some 
systems employ both the reference counter and the garbage collection schemes, 
besides allowing the programmer to erase nodes explicitly. The idea is to employ 
garbage collection only as a ‘last resort whenever all other methods of returning 
cells have failed. An elaborate system, which implements this idea and also 
includes a mechanism for postponing operations on reference counts in order to 
achieve further efficiency, has been described by L. P. Deutsch and D. G. Bobrow 
in CACM 19 (1976), 522-526. 

A sequential representation of Lists, which saves many of the link fields at 
the expense of more complicated storage management, is also possible. See N. E. 
Wiseman and J. O. Hiles, Comp. J. 10 (1968), 338-343; W. J. Hansen, CACM 
12 (1969), 499-507; and C. J. Cheney, CACM 13 (1970), 677-678. 

Daniel P. Friedman and David S. Wise have observed that the reference 
counter method can be employed satisfactorily in many cases even when Lists 
point to themselves, if certain link fields are not included in the counts [Inf. Proc. 
Letters 8 (1979), 41-45]. 

A great many variants and refinements of garbage collection algorithms 
have been proposed. Jacques Cohen, in Computing Surveys 13 (1981), 341- 
367, presents a detailed review of the literature prior to 1981, with important 
comments about the extra cost of memory accesses when pages of data are 
shuttled between slow memory and fast memory. 

Garbage collection as we have described it is unsuitable for “real time” 
applications, where each basic List operation must be quick; even if the garbage 
collector goes into action infrequently, it requires large chunks of computer time 
on those occasions. Exercise 12 discusses some approaches by which real-time 
garbage collection is possible. 
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It is a very sad thing nowadays 
that there is so little useless information. 

— OSCAR WILDE (1894) 

EXERCISES 

► 1. [M21] In Section 2.3.4 we saw that trees are special cases of the “classical” 
mathematical concept of a directed graph. Can Lists be described in graph-theoretic 
terminology? 

2. [ 20 ] In Section 2.3.1 we saw that tree traversal can be facilitated using a threaded 
representation inside the computer. Can List structures be threaded in an analogous 
way? 

3. [M26] Prove the validity of Algorithm E. [Hint: See the proof of Algorithm 
2.3. IT.] 

4. [28] Write a MIX program for Algorithm E, assuming that nodes are represented as 

one MIX word, with MARK the (0:0) field [“+” = 0, = 1], ATOM the (1 : 1) field. ALINK 

the (2:3) field, BLINK the (4:5) field, and A = 0. Also determine the execution time 
of your program in terms of relevant parameters. (In the MIX computer the problem 
of determining whether a memory location contains -0 or +0 is not quite trivial, and 
this can be a factor in your program.) 

5. [25] (Schorr and Waite.) Give a marking algorithm that combines Algorithms B 
and E as follows: The assumptions of Algorithm E with regard to fields within the 
nodes, etc., are retained; but an auxiliary stack STACK [1], STACK [2], ..., STACK [N] is 
used as in Algorithm B, and the mechanism of Algorithm E is employed only when the 
stack is full. 

6. [00] The quantitative discussion at the end of this section says that the cost of 
garbage collection is approximately ciN + C 2 M units of time; where does the “C 2 M” term 
come from? 

7. [24] (R. W. Floyd.) Design a marking algorithm that is similar to Algorithm E in 

using no auxiliary stack, except that (i) it has a more difficult task to do, because each 
node contains only MARK, ALINK, and BLINK fields- there is no ATOM field to provide 
additional control; yet (ii) it has a simpler task to do, because it marks only a binary 
tree instead of a general List. Here ALINK and BLINK are the usual LLINK and RLINK in 
a binary tree. 

► 8. [27] (L. P. Deutsch.) Design a marking algorithm similar to Algorithms D and E 
in that it uses no auxiliary memory for a stack, but modify the method so that it 
works with nodes of variable size and with a variable number of pointers having the 
following format: The first word of a node has two fields MARK and SIZE; the MARK field 
is to be treated as in Algorithm E, and the SIZE field contains a number n > 0. This 
means that there are n consecutive words after the first word, each containing two 
fields MARK (which is zero and should remain so) and LINK (which is A or points to the 
first word of another node). For example, a node with three pointers would comprise 
four consecutive words: 

First word MARK = 0 (will be set to 1) SIZE = 3 

Second word MARK = 0 LINK = first pointer 

Third word MARK = 0 LINK = second pointer 

Fourth word MARK = 0 LINK = third pointer. 

Your algorithm should mark all nodes reachable from a given node P0. 
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► 9. [28] (D. Edwards.) Design an algorithm for the second phase of garbage collection 
that “compacts storage” in the following sense: Let N0DE(1), . . . , NODE(M) be one-word 
nodes with fields MARK, ATOM, ALINK, and BLINK, as described in Algorithm E. Assume 
that MARK = 1 in all nodes that are not garbage. The desired algorithm should relocate 
the marked nodes, if necessary, so that they all appear in consecutive locations N0DE(1) , 
.... NODE(K) , and at the same time the ALINK and BLINK fields of nonatomic nodes 
should be altered if necessary so that the List structure is preserved. 

► 10. [28] Design an algorithm that copies a List structure, assuming that an internal 
representation like that in ( 7 ) is being used. (Thus, if your procedure is asked to copy 
the List whose head is the node at the upper left corner of ( 7 ), a new set of Lists having 
14 nodes, and with structure and information identical to that shown in ( 7 ), should 
be created.) 

Assume that the List structure is stored in memory using S, T, REF, and RLINK 
fields as in ( 9 ), and that NODE(PO) is the head of the List to be copied. Assume further 
that the REF field in each List head node is A; to avoid the need for additional memory 
space, your copying procedure should make use of the REF fields (and reset them to A 
again afterwards). 

11 . [M30] Any List structure can be “fully expanded” into a tree structure by repeat- 
ing all overlapping elements until none are left; when the List is recursive, this gives an 
infinite tree. For example, the List ( 5 ) would expand into an infinite tree whose first 
four levels are 


* 



/\ /\ /\ 1 //\\ 

h* j* f * g* /* g* d* a* b c* e* 

/l\\/\ A AAA I A 

Design an algorithm to test the equivalence of two List structures, in the sense 
that they have the same diagram when fully expanded. For example. Lists A and B 
are equivalent in this sense, if 

A = (a: C, b, a : ( b : D)) 

B = (a: ( b : D), b, a: E) 

C= ( b : (a: C)) 

D = (a:(b: D)) 

E= ( b:(a:C )). 

12 . [30] (M. Minsky.) Show that it is possible to use a garbage collection method 
reliably in a “real time” application, for example when a computer is controlling 
some physical device, even when stringent upper bounds are placed on the maximum 
execution time required for each List operation performed. [Hint: Garbage collection 
can be arranged to work in parallel with the List operations, if appropriate care is 
taken.] 
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2.4. MULTILINKED STRUCTURES 

Now THAT WE have examined linear lists and tree structures in detail, the 
principles of representing structural information within a computer should be 
evident. In this section we will look at another application of these techniques, 
this time for the typical case in which the structural information is slightly more 
complicated: In higher-level applications, several types of structure are usually 
present simultaneously. 

A “multilinked structure” involves nodes with several link fields in each 
node, not just one or two as in most of our previous examples. We have already 
seen some examples of multiple linkage, such as the simulated elevator system 
in Section 2.2.5 and the multivariate polynomials in Section 2.3.3. 

We shall see that the presence of many different kinds of links per node does 
not necessarily make the accompanying algorithms any more difficult to write 
or to understand than the algorithms already studied. We will also discuss the 
important question, “ How much structural information ought to be explicitly 
recorded in memory?” 

The problem we will consider arises in connection with writing a compiler 
program for the translation of COBOL and related languages. A programmer who 
uses COBOL may give alphabetic names to program variables on several levels; 
for example, the program might refer to files of data for sales and purchases! 
having the following structure: 

1 SALES 1 PURCHASES 


DATE 

2 

DATE 

3 MONTH 


3 DAY 

3 DAY 


3 MONTH 

3 YEAR 


3 YEAR 

TRANSACTION 

2 

TRANSACTION 

3 ITEM 


3 ITEM 

3 QUANTITY 


3 QUANTITY 

3 PRICE 


3 PRICE 

3 TAX 


3 TAX 

3 BUYER 


3 SHIPPER 

4 NAME 


4 NAME 

4 ADDRESS 


4 ADDRESS 


This configuration indicates that each item in SALES consists of two parts, the 
DATE and the TRANSACTION; the DATE is further divided into three parts, and the 
TRANSACTION likewise has five subdivisions. Similar remarks apply to PURCHASES. 
The relative order of these names indicates the order in which the quantities 
appear in external representations of the file (for example, magnetic tape or 
Prmted forms); notice that in this example “DAY” and “MONTH” appear in opposite 
order in the two files. The programmer also gives further information, not shown 
in this illustration, that tells how much space each item of information occupies 
and in what format it appears; such considerations are not relevant to us in this 
section, so they will not be mentioned further. 
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A COBOL programmer first describes the file layout and the other program 
variables, then specifies the algorithms that manipulate those quantities. To 
refer to an individual variable in the example above, it would not be sufficient 
merely to give the name DAY, since there is no way of telling if the variable called 
DAY is in the SALES file or in the PURCHASES file. Therefore a COBOL progr amm er 
is given the ability to write “DAY OF SALES” to refer to the DAY part of a SALES 
item. The programmer could also write, more completely, 

“DAY OF DATE OF SALES”, 

but in general there is no need to give more qualification than necessary to avoid 
ambiguity. Thus, 

“NAME OF SHIPPER OF TRANSACTION OF PURCHASES” 

may be abbreviated to 

“NAME OF SHIPPER” 

since only one part of the data has been called SHIPPER. 

These rules of COBOL may be stated more precisely as follows: 

a) Each name is immediately preceded by an associated positive integer called 
its level number. A name either refers to an elementary item or it is the 
name of a group of one or more items whose names follow. In the latter 
case, each item of the group must have the same level number, which must 
be greater than the level number of the group name. (For example, DATE 
and TRANSACTION above have level number 2, which is greater than the level 
number 1 of SALES.) 

b) To refer to an elementary item or group of items named A 0 , the general 
form is 

A 0 OF Ax OF . . . OF A n , 

where n > 0 and where, for 0 < j < n, Aj is the name of some item 
contained directly or indirectly within a group named A J+1 . There must be 
exactly one item Ao satisfying this condition. 

c) If the same name Aq appears in several places, there must be a way to refer 
to each use of the name by using qualification. 

As an example of rule (c), the data configuration 

1 AA 
2 BB 

3 CC ( 2 ) 

3 DD 
2 CC 

would not be allowed, since there is no unambiguous way to refer to the second 
appearance of CC. (See exercise 4.) 
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COBOL has another feature that affects compiler writing and the application 
we are considering, namely an option in the language that makes it possible to 
refer to many items at once. A COBOL programmer may write 

MOVE CORRESPONDING a TO /3 

which moves all items with corresponding names from data area a to data area / 3 . 
For example, the COBOL statement 

MOVE CORRESPONDING DATE OF SALES TO DATE OF PURCHASES 

would mean that the values of MONTH, DAY, and YEAR from the SALES file are to 
be moved to the variables MONTH, DAY, and YEAR in the PURCHASES file. (The 
relative order of DAY and MONTH is thereby interchanged.) 

The problem we will investigate in this section is to design three algorithms 
suitable for use in a COBOL compiler, which are to do the following things: 

Operation 1 . To process a description of names and level numbers such 
as (i), putting the relevant information into tables within the compiler for use 
in operations 2 and 3. 

Operation 2. To determine if a given qualified reference, as in rule (b), is 
valid, and when it is valid to locate the corresponding data item. 

Operation 3. To find all corresponding pairs of items indicated by a given 
CORRESPONDING statement. 

We will assume that our compiler already has a “symbol table subroutine” 
that will convert an alphabetic name into a link that points to a table entry for 
that name. (Methods for constructing symbol table algorithms are discussed in 
detail in Chapter 6 .) In addition to the Symbol Table, there is a larger table 
that contains one entry for each item of data in the COBOL source program that 
is being compiled; we will call this the Data Table. 

Clearly, we cannot design an algorithm for operation 1 until we know what 
kind of information is to be stored in the Data Table, and the form of the Data 
Table depends on what information we need in order to perform operations 2 
and 3; thus we look first at operations 2 and 3. 

In order to determine the meaning of the COBOL reference 

A 0 OF Aj OF . . . OF A n , n > 0, ( 3 ) 

we should first look up the name A 0 in the Symbol Table. There ought to be 
a series of links from the Symbol Table entry to all Data Table entries for this 
name. Then for each Data Table entry we will want a link to the entry for the 
group item that contains it. Now if there is a further link field from the Data 
Table items back to the Symbol Table, it is not hard to see how a reference like 
( 3 ) can be processed. Furthermore, we will want some sort of links from the 
Data Table entries for group items to the items in the group, in order to locate 
the pairs indicated by “MOVE CORRESPONDING” . 
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We have thereby found a potential need for five link fields in each Data 
Table entry: 

PREV (a link to the previous entry with the same name, if any); 

PARENT (a link to the smallest group, if any, containing this item); 

NAME (a link to the Symbol Table entry for this item); 

CHILD (a link to the first subitem of a group); 

SIB (a link to the next subitem in the group containing this item). 

It is clear that COBOL data structures like those for SALES and PURCHASES above 
are essentially trees; and the PARENT, CHILD, and SIB links that appear here are 
familiar from our previous study. (The conventional binary tree representation 
of a tree consists of the CHILD and SIB links; adding the PARENT link gives what 
we have called a “triply linked tree.” The five links above consist of these three 
tree links together with PREV and NAME, which superimpose further information 
on the tree structure.) 

Perhaps not all five of these links will turn out to be necessary, or sufficient, 
but we will try first to design our algorithms under the tentative assumption 
that Data Table entries will involve these five link fields (plus further information 
irrelevant to our problems). As an example of the multiple linking used, consider 
the two COBOL data structures 

1 A 1 H 

3 B 5 F 

7 C 8 G 

7 D 5 B 

3 E 5 C 

3 F 9 E 

4 G 9 D 

9 G 

They would be represented as shown in (5) (with links indicated symbolically). 
The LINK field of each Symbol Table entry points to the most recently encoun- 
tered Data Table entry for the symbolic name in question. 

The first algorithm we require is one that builds the Data Table in such 
a form. Note the flexibility in choice of level numbers that is allowed by the 
COBOL rules; the left structure in (4) is completely equivalent to 

1 A 
2 B 
3 C 
3 D 
2 E 
2 F 
3 G 

because level numbers do not have to be sequential. 
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Symbol Table 


Data Table 



Empty boxes indicate 
additional information 
not relevant here 



PREV 

PARENT 

NAME 

CHILD 

SIB 

Al: 

A 

A 

A 

B3 

HI 


B3: 

A 

Al 

B 

C7 

E3 


C7: 

A 

B3 

C 

A 

D7 


D7: 

A 

B3 

D 

A 

A 


E3: 

A 

Al 

E 

A 

F3 


F3: 

A 

Al 

F 

G4 

A 


G4: 

A 

F3 

G 

A 

A 


HI: 

A 

A 

H 

F5 

A 


F5: 

F3 

HI 

F 

G8 

B5 


G8: 

G4 

F5 

G 

A 

A 


B5: 

B3 

HI 

B 

A 

C5 


C5: 

C7 

HI 

C 

E9 

A 


E9: 

E3 

C5 

E 

A 

D9 


D9: 

D7 

C5 

D 

A 

G9 


G9: 

G8 

C5 

G 

A 

A 



(5) 


Some sequences of level numbers are illegal, however; for example, if the 
level number of D in (4) were changed to “6” (in either place) we would have 
a meaningless data configuration, violating the rule that all items of a group 
must have the same number. The following algorithm therefore makes sure that 
COBOL’s rule (a) has not been broken. 

Algorithm A ( Build Data Table). This algorithm is given a sequence of pairs 
(L,P), where L is a positive integer “level number” and P points to a Symbol 
Table entry, corresponding to COBOL data structures such as (4) above. The 
algorithm builds a Data Table as in the example (5) above. When P points to a 
Symbol Table entry that has not appeared before, LINK(P) will equal A. This 
algorithm uses an auxiliary stack that is treated as usual (using either sequential 
memory allocation, as in Section 2.2.2, or linked allocation, as in Section 2.2.3). 

Al. [Initialize.] Set the stack contents to the single entry (0, A). (The stack 
entries throughout this algorithm are pairs (L,P), where L is an integer and 
P a pointer; as this algorithm proceeds, the stack contains the level number 
and pointers to the most recent data entries on all levels higher in the tree 
than the current level. For example, just before encountering the pair “3 F” 
in the example above, the stack would contain 


(0, A) (1, Al) (3, E3) 


from bottom to top.) 
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A2. [Next item.] Let (L, P) be the next data item from the input. If the input 
is exhausted, however, the algorithm terminates. Set Q <= AVAIL (that is, 
let Q be the location of a new node in which we can put the next Data Table 
entry) . 

A3. [Set name links.] Set 

PREV(Q) 4- LINK (P) , LINK(P) 4- Q, NAME(Q) 4- P. 

(This properly sets two of the five links in NODE(Q). We now want to set 
PARENT, CHILD, and SIB appropriately.) 

A4. [Compare levels.] Let the top entry of the stack be (LI, PI). If LI < L, set 
CHILD(Pl) 4— Q (or, if PI = A, set FIRST 4— q, where FIRST is a variable 
that will point to the first Data Table entry) and go to A6. 

A5. [Remove top level.] If LI > L, remove the top stack entry, let (LI, PI) be the 
new entry that has just come to the top of the stack, and repeat step A5. If 
LI < L, signal an error (mixed numbers have occurred on the same level). 
Otherwise, namely when LI = L, set SIB (PI) 4— q, remove the top stack 
entry, and let (LI, PI) be the pair that has just come to the top of the stack. 
A6. [Set family links.] Set PARENT (q) 4- PI, CHILD (q) 4 - A, SIB(q) 4 - A. 

A7. [Add to stack.] Place (L, q) on top of the stack, and return to step A2. | 

The introduction of an auxiliary stack, as explained in step Al, makes this 
algorithm so transparent that it needs no further explanation. 

The next problem is to locate the Data Table entry corresponding to a 
reference 

A 0 OF Ai OF . . . OF A n , n > 0. (6) 

A good compiler will also check to ensure that such a reference is unambiguous. 
In this case, a suitable algorithm suggests itself immediately: All we need to do 
is to run through the list of Data Table entries for the name Aq and make sure 
that exactly one of these entries matches the stated qualification A 1: . .. ,A n . 

Algorithm B ( Check a qualified reference). Corresponding to reference (6), a 
Symbol Table subroutine will find pointers P 0 , P u . . . , P n to the Symbol Table 
entries for A 0 , Aj, . . . , A ra , respectively. 

The purpose of this algorithm is to examine P 0 , P 1 , . . . , P n and either to 
determine that reference (6) is in error, or to set variable q to the address of the 
Data Table entry for the item referred to by (6). 

Bl. [Initialize.] Set q 4- A, P 4- LINK(P 0 )- 

B2. [Done?] If P = A, the algorithm terminates; at this point q will equal A if 
(6) does not correspond to any Data Table entry. But if P A, set S 4 - P 
and k 4 — 0. (S is a pointer variable that will run from P up the tree through 
PARENT links; k is an integer variable that goes from 0 to n. In practice, the 
pointers P 0 , . . . , P n would often be kept in a linked list, and instead of k, we 
would substitute a pointer variable that traverses this list; see exercise 5.) 
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B3. [Match complete?] If k < n go on to B4. Otherwise we have found a 
matching Data Table entry; if Q / A, this is the second entry found, so an 
error condition is signaled. Set Q f- P, P f- PREV(P), and go to B2. 

B4. [Increase fc.] Set k <— k + 1. 

B5. [Move up tree.] Set S 4— PARENT (S). If S = A, we have failed to find a 
match; set P 4- PREV(P) and go to B2. 

B6. [Ak match?] If NAME(S) = Pk, go to B3, otherwise go to B5. | 

Note that the CHILD and SIB links are not needed by this algorithm. 



Fig. 40. Algorithm for checking a COBOL reference. 

The third and final algorithm that we need concerns “MOVE CORRESPONDING”; 
before we design such an algorithm, we must have a precise definition of what is 
required. The COBOL statement 

MOVE CORRESPONDING a TO f3 (7) 

where a and f3 are references such as (6) to data items, is an abbreviation for 
the set of all statements 

MOVE a' TO f3' 

where there exists an integer n > 0 and n names Ao,Ai,...,A n -i such that 

a 1 - A 0 OF Ai OF . . . OF A n _ 1 OF a 

f3’ = A 0 OF OF . . . OF A n _i OF /? V ; 

and either a! or f3' is an elementary item (not a group item). Furthermore we 
require that the first levels of (8) show complete qualifications, namely that Aj + 1 
be the parent of Aj for 0 < j < n; a' and (3 1 must be exactly n levels farther 
down in the tree than a and /3 are. 

With respect to our example (4), 

MOVE CORRESPONDING A TO H 
is therefore an abbreviation for the statements 
MOVE B OF A TO B OF H 

MOVE G OF F OF A TO G OF F OF H 

The algorithm to recognize all corresponding pairs a', (3' is quite interesting 
although not difficult; we move through the tree whose root is a, in preorder, 
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simultaneously looking in the f3 tree for matching names, and skipping over 
subtrees in which no corresponding elements can possibly occur. The names 
A 0 , . . . , A n _! of (8) are discovered in the opposite order A n _ 1 , . . . , A 0 . 

Algorithm C ( Find CORRESPONDING pairs). Given PO and QO, which point to 
Data Table entries for a and (3, respectively, this algorithm successively finds 
all pairs (P, Q) of pointers to items ( a',/3 ') satisfying the constraints mentioned 
above. 

Cl. [Initialize.] Set P 4- PO, Q 4— QO. (In the remainder of this algorithm, the 
pointer variables P and Q will walk through trees having the respective roots 
a and f3.) 

C2. [Elementary?] If CHILD (P) = A or CHILD (Q) = A, output (P, Q) as one of the 
desired pairs and go to C5. Otherwise set P 4- CHILD (P), Q 4- CHILD (Q). 
(In this step, P and Q point to items a 1 and (3 1 satisfying (8), and we wish to 
MOVE a TO 8 if and only if either a' or 8' (or both) is an elementary item.) 

C3. [Match name.] (Now P and Q point to data items that have respective 
qualifications of the forms 

Ao OF Ai OF . . . OF A n _ 1 OF a 

and 

B 0 OF A 1 OF . . . OF A n _! OF /3. 

The object is to see if we can make B 0 = Aq by examining all the names 
of the group Aj OF ... OF A n _j OF 8.) If NAME(P) = NAME(Q), go to C2 
(a match has been found). Otherwise, if SIB(Q) ^ A, set Q 4 - SIB(Q) and 
repeat step C3. (If SIB(Q) = A, no matching name is present in the group, 
and we continue on to step C4.) 

C4. [Move on.] If SIB(P) ^ A, set P 4- SIB(P) and Q 4- CHILD (PARENT (Q) ) , 
and go back to C3. If SIB(P) = A, set P 4- PARENT (P) and Q 4- PARENT (Q) . 

C5. [Done?] If P = PO, the algorithm terminates; otherwise go to C4. | 

A flow chart for this algorithm is shown in Fig. 41. A proof that this algorithm 
is valid can readily be constructed by induction on the size of the trees involved 
(see exercise 9). 



Fig. 41. Algorithm for “MOVE CORRESPONDING”. 
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At this point it is worthwhile to study the ways in which the five link fields 
PREV, PARENT, NAME, CHILD, and SIB are used by Algorithms B and C. The 
striking feature is that these five links constitute a “complete set” in the sense 
that Algorithms B and C do virtually the minimum amount of work as they 
move through the Data Table. Whenever they need to refer to another Data 
Table entry, its address is immediately available; there is no need to conduct a 
search. It would be difficult to imagine how Algorithms B and C could possibly 
be made any faster if any additional link information were present in the table. 
(See exercise 11, however.) 

Each link field may be viewed as a clue to the program, planted there in 
order to make the algorithms run faster. (Of course, the algorithm that builds 
the tables, Algorithm A, runs correspondingly slower, since it has more links to 
fill in. But table-building is done only once.) It is clear, on the other hand, that 
the Data Table constructed above contains much redundant information. Let us 
consider what would happen if we were to delete certain of the link fields. 

The PREV link, while not used in Algorithm C, is extremely important for 
Algorithm B, and it seems to be an essential part of any COBOL compiler unless 
lengthy searches are to be carried out. A field that links together all items of 
the same name therefore seems essential for efficiency. We could perhaps modify 
the strategy slightly and adopt circular linking instead of terminating each list 
with A, but there is no reason to do this unless other link fields are changed or 
eliminated. 

The PARENT link is used in both Algorithms B and C, although its use in 
Algorithm C could be avoided if we used an auxiliary stack in that algorithm, or 
if we augmented SIB so that thread links are included (as in Section 2.3.2). So we 
see that the PARENT link has been used in an essential way only in Algorithm B. If 
the SIB link were threaded, so that the items that now have SIB = A would have 
SIB = PARENT instead, it would be possible to locate the parent of any data item 
by following the SIB links; the added thread links could be distinguished either 
by having a new TAG field in each node that says whether the SIB link is a thread, 
or by the condition “SIB(P) < P ’ if the Data Table entries are kept consecutively 
in memory in order of appearance. This would mean a short search would be 
necessary in step B5, and the algorithm would be correspondingly slower. 

The NAME link is used by the algorithms only in steps B6 and C3. In both 
cases we could make the tests “NAME(S) = P k " and “NAME(P) = NAME(Q)” in 
other ways if the NAME link were not present (see exercise 10), but this would 
significantly slow down the inner loops of both Algorithms B and C. Here again 
we see a trade-off between the space for a link and the speed of the algorithms. 
(The speed of Algorithm C is not especially significant in COBOL compilers, when 
typical uses of MOVE CORRESPONDING are considered; but Algorithm B should be 
fast.) Experience indicates that other important uses are found for the NAME link 
within a COBOL compiler, especially in printing diagnostic information. 

Algorithm A builds the Data Table step by step, and it never has occasion 
to return a node to the pool of available storage; so we usually find that Data 
Table entries take consecutive memory locations in the order of appearance of 
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the data items in the COBOL source program. Thus in our example (5), locations 
Al, B3, . . . would follow each other. This sequential nature of the Data Table 
leads to certain simplifications; for example, the CHILD link of each node is 
either A or it points to the node immediately following, so CHILD can be reduced 
to a 1-bit field. Alternatively, CHILD could be removed in favor of a test if 
PARENT (P + c) = P, where c is the node size in the Data Table. 

Thus the five link fields are not all essential, although they are helpful from 
the standpoint of speed in Algorithms B and C. This situation is fairly typical 
of most multilinked structures. 

It is interesting to note that at least half a dozen people writing COBOL 
compilers in the early 1960s arrived independently at this same way to maintain 
a Data Table using five links (or four of the five, usually with the CHILD link 
missing). The first publication of such a technique was by H. W. Lawson, Jr. 
[ACM National Conference Digest (Syracuse, N.Y.: 1962)]. But in 1965 an 
ingenious technique for achieving the effects of Algorithms B and C, using only 
two link fields and sequential storage of the Data Table, without a very great 
decrease in speed, was introduced by David Dahrn; see exercises 12 through 14. 

EXERCISES 

1. [ 00 ] Considering COBOL data configurations as tree structures, are the data items 
listed by a COBOL programmer in preorder, postorder, or neither of those orders? 

2. [10] Comment about the running time of Algorithm A. 

3. [ 22 ] The PL/I language accepts data structures like those in COBOL, except that 
any sequence of level numbers is possible. For example, the sequence 


1 A 1 A 

3 B 2 B 

5 C is equivalent to 3 C 

4 D 3 D 

2 E 2 E 


In general, rule (a) is modified to read, “The items of a group must have a sequence 
of nonincreasing level numbers, all of which are greater than the level number of the 
group name.” What modifications to Algorithm A would change it from the COBOL 
convention to this PL/I convention? 

► 4. [ 26 ] Algorithm A does not detect the error if a COBOL programmer violates rule 

(c) stated in the text. How should Algorithm A be modified so that only data structures 
satisfying rule (c) will be accepted? 

5. [ 20 ] In practice, Algorithm B may be given a linked list of Symbol Table references 
as input, instead of what we called “Po, Pi, ... , P n .” Let T be a pointer variable such 
that 

INFO (T) = Po, INFO (RLINK(T) ) =P\, ..., INFO (RLINK W (T) ) = P n , RLINK ln+1] (T) = A. 

Show how to modify Algorithm B so that it uses such a linked list as input. 

6. [23] The PL/I language accepts data structures much like those in COBOL, but 
does not make the restriction of rule (c); instead, we have the rule that a qualified 
reference (3) is unambiguous if it shows “complete” qualification — that is, if A j+l is 
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the parent of Aj for 0 < j < n, and if A n has no parent. Rule (c) is now weakened 
to the simple condition that no two items of a group may have the same name. The 
second CC in (2) would be referred to as “CC OF AA” without ambiguity; the three 
data items 

1 A 
2 A 
3 A 

would be referred to as “A”, “A OF A”, “A OF A OF A” with respect to the PL/I con- 
vention just stated. [Note: Actually the word “OF” is replaced by a period in PL/I, 
and the order is reversed; “CC OF AA” is really written “AA.CC” in PL/I, but this is not 
important for the purposes of the present exercise.] Show how to modify Algorithm B 
so that it follows the PL/I convention. 

7. [15] Given the data structures in ( 1 ), what does the COBOL statement “MOVE 
CORRESPONDING SALES TO PURCHASES” mean? 

8. [10] Under what circumstances is “MOVE CORRESPONDING a TO /3” exactly the 
same as “MOVE a TO /3”, according to the definition in the text? 

9. [M23] Prove that Algorithm C is correct. 

10. [23] (a) How could the test “NAME(S) = P*.” in step B6 be performed if there were 
no NAME link in the Data Table nodes? (b) How could the test “NAME(P) = NAME(Q)” 
in step C3 be performed if there were no NAME link in the Data Table entries? (Assume 
that all other links are present as in the text.) 

► 11. [23] What additional links or changes in the strategy of the algorithms of the text 
could make Algorithm B or Algorithm C faster? 

12 . [25] (D. M. Dahm.) Consider representing the Data Table in sequential locations 
with just two links for each item: 

PREV (as in the text); 

SCOPE (a link to the last elementary item in this group). 

We have SCOPE(P) = P if and only if NODE(P) represents an elementary item. For 
example, the Data Table of ( 5 ) would be replaced by 



PREV 

SCOPE 


PREV 

SCOPE 


PREV 

SCOPE 

A1 

A 

G4 

F3: 

A 

G4 

B5 

B3 

B5 

B3 

A 

D7 

G4: 

A 

G4 

C5 

C7 

G9 

C7 

A 

C7 

HI: 

A 

G9 

E9 

E3 

E9 

D7 

A 

D7 

F5: 

F3 

G8 

D9 

D7 

D9 

E3 

A 

E3 

G8: 

G4 

G8 

G9 

G8 

G9 


(Compare with (5) of Section 2.3.3.) Notice that NODE(P) is part of the tree below 
NODE(Q) if and only if Q < P < SCOPE(Q). Design an algorithm that performs the 
function of Algorithm B when the Data Table has this format. 

► 13. [24] Give an algorithm to substitute for Algorithm A when the Data Table is to 
have the format shown in exercise 12. 

► 14. [28] Give an algorithm to substitute for Algorithm C when the Data Table has 
the format shown in exercise 12. 

15. [25] (David S. Wise.) Reformulate Algorithm A so that no extra storage is used 
for the stack. [Hint: The SIB fields of all nodes pointed to by the stack are A in the 
present formulation.] 
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2.5. DYNAMIC STORAGE ALLOCATION 

We have SEEN how the use of links implies that data structures need not be 
sequentially located in memory; a number of tables may independently grow and 
shrink in a common pooled memory area. However, our discussions have always 
tacitly assumed that all nodes have the same size — that every node occupies a 
certain fixed number of memory cells. 

For a great many applications, a suitable compromise can be found so that 
a uniform node size is indeed used for all structures (for example, see exercise 2). 
Instead of simply taking the maximum size that is needed and wasting space in 
smaller nodes, it is customary to pick a rather small node size and to employ 
what may be called the classical linked-memory philosophy : “If there isn’t room 
for the information here, let’s put it somewhere else and plant a link to it.” 

For a great many other applications, however, a single node size is not 
reasonable; we often wish to have nodes of varying sizes sharing a common 
memory area. Putting this another way, we want algorithms for reserving and 
freeing variable-size blocks of memory from a larger storage area, where these 
blocks are to consist of consecutive memory locations. Such techniques are 
generally called dynamic storage allocation algorithms. 

Sometimes, often in simulation programs, we want dynamic storage alloca- 
tion for nodes of rather small sizes (say one to ten words); and at other times, 
often in operating systems, we are dealing primarily with rather large blocks of 
information. These two points of view lead to slightly different approaches to 
dynamic storage allocation, although the methods have much in common. For 
uniformity in terminology between these two approaches, we will generally use 
the terms block and area rather than “node” in this section, to denote a set of 
contiguous memory locations. 

Several authors began about 1975 to call the pool of available memory a 
“heap.” But in the present series of books, we will use that word only in its 
more traditional sense related to priority queues (see Section 5.2.3). 

A. Reservation. Figure 42 shows a typical memory map or “checkerboard,” a 
chart showing the current state of some memory pool. In this case the memory is 
shown partitioned into 53 blocks of storage that are “reserved,” or in use, mixed 
together with 21 “free” or “available” blocks that are not in use. After dynamic 
storage allocation has been in operation for awhile, the computer memory will 
perhaps look something like this. Our first problem is to answer two questions: 

a) How is this partitioning of available space to be represented inside the 
computer? 

b) Given such a representation of the available spaces, what is a good algorithm 
for finding a block of n consecutive free spaces and reserving them? 

The answer to question (a) is, of course, to keep a list of the available space 
somewhere; this is almost always done best by using the available space itself to 
contain such a list. (An exception is the case when we are allocating storage for 
a disk file or other memory in which nonuniform access time makes it better to 
maintain a separate directory of available space.) 
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Fig. 42. A memory map. 


Thus, we can link together the available segments: The first word of each 
free storage area can contain the size of that block and the address of the next 
free area. The free blocks can be linked together in increasing or decreasing order 
of size, or in order of memory address, or in essentially random order. 

For example, consider Fig. 42, which illustrates a memory of 131,072 words, 
addressed from 0 to 131071. If we were to link together the available blocks in 
order of memory location, we would have one variable AVAIL pointing to the 
first free block (in this case AVAIL would equal 0), and the other blocks would 


be represented as 

follows: 



location 

SIZE 

LINK 


0 

101 

632 


632 

42 

1488 


73654 

1909 

77519 

[17 similar entries] 

77519 

53553 

A 

[special marker for last link] 

Thus locations 0 

through 100 form the 

first available block; after the reserved 


areas 101-290 and 291-631 shown in Fig. 42, we have more free space in location 
632-673; etc. 

As for question (b), if we want n consecutive words, clearly we must locate 
some block of m > n available words and reduce its size to m - n. (Furthermore, 
when m — n, we must also delete this block from the list.) There may be several 
blocks with n or more cells, and so the question becomes, which area should be 
chosen? 

Two principal answers to this question suggest themselves: We can use the 
best-fit method or the first-fit, method. In the former case, we decide to choose 
an area with m cells, where m is the smallest value present that is n or more. 
This might require searching the entire list of available space before a decision 
can be made. The first-fit method, on the other hand, simply chooses the first 
area encountered that has > n words. 

Historically, the best-fit method was widely used for several years; this 
naturally appears to be a good policy since it saves the larger available areas 


2.5 


DYNAMIC STORAGE ALLOCATION 437 


for a later time when they might be needed. But several objections to the best- 
fit technique can be raised: It is rather slow, since it involves a fairly long search; 
if best-fit is not substantially better than first-fit for other reasons, this extra 
searching time is not worthwhile. More importantly, the best-fit method tends 
to increase the number of very small blocks, and proliferation of small blocks is 
usually undesirable. There are certain situations in which the first-fit technique 
is demonstrably better than the best-fit method; for example, suppose we are 
given just two available areas of memory, of sizes 1300 and 1200, and suppose 
there are subsequent requests for blocks of sizes 1000, 1100, and 250: 


memory 

available areas, 

available areas, 

request 

first-fit 

best-fit 

— 

1300, 1200 

1300, 1200 

1000 

300, 1200 

1300, 200 

1100 

300, 100 

200, 200 

250 

50, 100 

stuck 


(A contrary example appears in exercise 7.) The point is that neither method 
clearly dominates the other, hence the simple first-fit method can be recom- 
mended. 

Algorithm A ( First-fit method). Let AVAIL point to the first available block 
of storage, and suppose that each available block with address P has two fields: 
SIZE(P) , the number of words in the block; and LINK(P), a pointer to the next 
available block. The last pointer is A. This algorithm searches for and reserves 
a block of N words, or reports failure. 

Al. [Initialize.] Set Q 4— LOC(AVAIL). (Throughout the algorithm we use two 
pointers, Q and P, which are generally related by the condition P = LINK(Q) . 
We assume that LINK (L0C (AVAIL) ) = AVAIL.) 

A2. [End of list?] Set P 4— LINK(Q). If P = A, the algorithm terminates 
unsuccessfully; there is no room for a block of N consecutive words. 

A3. [Is SIZE enough?] If SIZE(P) > N, go to A4; otherwise set Q 4— P and return 
to step A2. 

A4. [Reserve N.] Set K 4- SIZE(P) - N. If K = 0, set LINK(Q) 4 - LINK(P) 
(thereby removing an empty area from the list); otherwise set SIZE(P) 4- K. 
The algorithm terminates successfully, having reserved an area of length N 
beginning with location P + K. | 

This algorithm is certainly straightforward enough. However, a significant 
improvement in its running speed can be made with only a rather slight change 
in strategy. This improvement is quite important, and the reader will find it a 
pleasure to discover it without being told the secret (see exercise 6). 

Algorithm A may be used whether storage allocation is desired for small N 
or large N. Let us assume temporarily, however, that we are primarily interested 
in large values of N. Then notice what happens when SIZE(P) is equal to N + 1 in 
that algorithm: We get to step A4 and reduce SIZE(P) to 1. In other words, an 
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available block of size 1 has just been created; this block is so small it is virtually 
useless, and it just clogs up the system. We would have been better off if we had 
reserved the whole block of N + 1 words, instead of saving the extra word; it is 
often better to expend a few words of memory to avoid handling unimportant 
details. Similar remarks apply to blocks of N + K words when K is very small. 

If we allow the possibility of reserving slightly more than N words it will be 
necessary to remember how many words have been reserved, so that later when 
this block becomes available again the entire set of N + K words is freed. This 
added amount of bookkeeping means that we are tying up space in every block 
m order to make the system more efficient only in certain circumstances when 
a tight fit is found; so the strategy doesn’t seem especially attractive. However, 
a special control word as the first word of each variable-size block often turns 
out to be desirable for other reasons, and so it is usually not unreasonable to 
expect the SIZE field to be present in the first word of all blocks, whether they 
are available or reserved. 

In accordance with these conventions, we would modify step A4 above to 
read as follows: 

A4. [Reserve > N.] Set K 4— SIZE(P) — N. If K < c (where c is a small positive 
constant chosen to reflect an amount of storage we are willing to sacrifice in 
the interests of saving time), set LINK(Q) 4- LINK(P) and L 4 — P. Otherwise 
set SIZE(P) 4- K, L 4- P + K, SIZE(L) 4- N. The algorithm terminates 
successfully, having reserved an area of length N or more beginning with 
location L. 

A value for the constant c of about 8 or 10 is suggested, although very little 
theory or empirical evidence exists to compare this with other choices. When 
the best-fit method is being used, the test of K < c is even more important than 
it is for the first-fit method, because tighter fits (smaller values of K) are much 
more likely to occur, and the number of available blocks should be kept as small 
as possible for that algorithm. 

B. Liberation. Now let’s consider the inverse problem: How should we return 
blocks to the available space list when they are no longer needed? 

It is perhaps tempting to dismiss this problem by using garbage collection 
(see Section 2.3.5); we could follow a policy of simply doing nothing until space 
runs out, then searching for all the areas currently in use and fashioning a new 
AVAIL list. 

The idea of garbage collection is not to be recommended, however, for all 
applications. In the first place, we need a fairly “disciplined” use of pointers if we 
are to be able to guarantee that all areas currently in use will be easy to locate, 
and this amount of discipline is often lacking in the applications considered here. 
Secondly, as we have seen before, garbage collection tends to be slow when the 
memory is nearly full. 

There is another more important reason why garbage collection is not satis- 
factory, due to a phenomenon that did not confront us in our previous discussion 
of the technique: Suppose that there are two adjacent areas of memory, both 
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of which are available, but because of the garbage-collection philosophy one of 
them (shown shaded) is not in the AVAIL list. 

( 2 ) 

In this diagram, the heavily shaded areas at the extreme left and right are 
unavailable. We may now reserve a section of the area known to be available: 

(3) 

If garbage collection occurs at this point, we have two separate free areas, 



Boundaries between available and reserved areas have a tendency to perpetuate 
themselves, and as time goes on the situation gets progressively worse. But if 
we had used a philosophy of returning blocks to the AVAIL list as soon as they 
become free, and collapsing adjacent available areas together , we would have 
collapsed (2) into 


I 




and we would have obtained 



(5) 

( 6 ) 


which is much better than (4). This phenomenon causes the garbage-collection 
technique to leave memory more broken up than it should be. 

In order to remove this difficulty, we can use garbage collection together 
with the process of compacting memory, that is, moving all the reserved blocks 
into consecutive locations, so that all available blocks come together whenever 
garbage collection is done. The allocation algorithm now becomes completely 
trivial by contrast with Algorithm A, since there is only one available block at all 
times. Even though this technique takes time to recopy all the locations that are 
in use, and to change the value of the link fields therein, it can be applied with 
reasonable efficiency when there is a disciplined use of pointers, and when there 
is a spare link field in each block for use by the garbage collection algorithms. 
(See exercise 33.) 

Since many applications do not meet the requirements for the feasibility of 
garbage collection, we shall now study methods for returning blocks of memory 
to the available space list. The only difficulty in these methods is the collapsing 
problem: Two adjacent free areas should be merged into one. In fact, when an 
area bounded by two available blocks becomes free, all three areas should be 
merged together into one. In this way a good balance is obtained in memory 
even though storage areas are continually reserved and freed over a long period 
of time. (For a proof of this fact, see the “fifty-percent rule” below.) 

The problem is to determine whether the areas at either side of the returned 
block are currently available; and if they are, we want to update the AVAIL list 
properly. The latter operation is a little more difficult than it sounds. 
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The first solution to these problems is to maintain the AVAIL list in order of 
increasing memory locations. 

Algorithm B ( Liberation with sorted list). Under the assumptions of Algo- 
rithm A, with the additional assumption that the AVAIL list is sorted by memory 
location (that is, if P points to an available block and LINK(P) / A. then 
LINK (P) > P), this algorithm adds the block of N consecutive cells beginning 
at location P0 to the AVAIL list. We naturally assume that none of these N cells 
is already available. 

Bl. [Initialize.] Set Q ■<— LOC(AVAIL). (See the remarks in step A1 above.) 

B2. [Advance P.] Set P «- LINK(Q) . If P = A, or if P > P0, go to B3; otherwise 
set Q P and repeat step B2. 

B3. [Check upper bound.] If P0 + N — P and P yt A, set N ■<— N + SIZE(P) and 
set LINK(PO) <— LINK(P). Otherwise set LINK(PO) P. 

B4. [Check lower bound.] If Q + SIZE(Q) = P0 (we assume that 

SIZE (LOC (AVAIL) ) = 0, 

so this test always fails when Q = LOC (AVAIL)), set SIZE(Q) SIZE(Q) + N 
andLINK(Q) t— LINK(PO). Otherwise set LINK (Q) -e-PO. SIZE(PO) i— N. | 

Steps B3 and B4 do the desired collapsing, based on the fact that the pointers 
Q < P0 < P are the beginning locations of three consecutive available areas. 

If the AVAIL list is not maintained in order of locations, the reader can see 
that a brute force approach to the collapsing problem would require a complete 
search through the entire AVAIL list; Algorithm B reduces this to a search through 
about half of the AVAIL list (in step B2) on the average. Exercise 11 shows how 
Algorithm B can be modified so that, on the average, only about one-third of the 
AVAIL list must be searched. But obviously, when the AVAIL list is long, all of 
these methods are much slower than we want them to be. Isn’t there some way 
to reserve and free storage areas so that we don’t need to do extensive searching 
through the AVAIL list? 

We will now consider a method that eliminates all searching when storage 
is returned and that can be modified, as in exercise 6, to avoid almost all of 
the searching when storage is reserved. The technique makes use of a TAG field 
at both ends of each block, and a SIZE field in the first word of each block: 
this overhead is negligible when reasonably large blocks are being used, although 
it is perhaps too much of a penalty to pay in situations when the blocks have a 
veiy small average size. Another method described in exercise 19 requires only 
one bit in the first word of each block, at the expense of a little more running 
time and a slightly more complicated program. 

At any rate, let us now assume that we don’t mind adding a little bit of 
control information, in order to save a good deal of time over Algorithm B when 
the AVAIL list is long. The method we will describe assumes that each block has 
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the following form: 

Reserved block (TAG = “+”) Free block (TAG = 

First word 
Second word 

SIZE- 2 words 

Last word 


- 

SIZE 


LINK 




LINK 


- 

SIZE 



+ 

SIZE 




+ 



(7) 


The idea in the following algorithm is to maintain a doubly linked AVAIL list, 
so that entries may conveniently be deleted from random parts of the list. The 
TAG field at either end of a block can be used to control the collapsing process, 
since we can tell easily whether or not both adjacent blocks are available. 

Double linking is achieved in a familiar way, by letting the LINK in the first 
word point to the next free block in the list, and letting the LINK in the second 
word point back to the previous block; thus, if P is the address of an available 
block, we always have 


LINK(LINK(P) + 1) = P = LINK (LINK (P + 1)). 

To ensure proper “boundary conditions,” the list head is set up as follows: 


LOCK AVAIL): 
LOC (AVAIL) +1: 


- 

' 0 

0 



- 

0 

0 




> to first block in available space list 

> to last block in available space list 


( 8 ) 


(9) 


A first-fit reservation algorithm for this technique may be designed very 
much like Algorithm A, so we shall not consider it here (see exercise 12). The 
principal new feature of this method is the way the block can be freed in 
essentially a fixed amount of time: 

Algorithm C ( Liberation with boundary tags). Assume that blocks of locations 
have the forms shown in (7), and assume that the AVAIL list is doubly linked, as 
described above. This algorithm puts the block of locations starting with address 
PO into the AVAIL list. If the pool of available storage runs from locations mo 
through mi, inclusive, the algorithm assumes for convenience that 

TAG(m 0 - 1) = TAG(m.! + 1) = “+”. 

Cl. [Check lower bound.] If TAG(PO — 1) = “+”, go to C3. 

C2. [Delete lower area.] Set P 4- P0-SIZE(P0 - 1), and then set PI 4- LINK(P) , 
P2 4- LINK(P + 1), LINK(P1 + 1) 4- P2, LINK(P2) 4- PI, SIZE(P) 4- 
SIZE(P) + SIZE(PO) . PO 4- P. 

C3. [Check upper bound.] Set P 4- PO + SIZE(PO). If TAG(P) = “+”, go to C5. 

C4. [Delete upper area.] Set PI 4- LINK(P) , P2 4- LINK (P + 1) , LINK (PI + 1) 4- 
P2, LINK(P2) 4- PI, SIZE(PO) 4- SIZE(PO) + SIZE(P), P 4- P + SIZE(P). 





442 INFORMATION STRUCTURES 


2.5 


C5. [Add to AVAIL list.] Set SIZE(P - 1) 4- SIZE(PO), LINK(PO) 4 - AVAIL. 
LINK (PO + 1) 4- LOC (AVAIL) , LINK (AVAIL + 1) 4 - PO. AVAIL 4 - PO. 
TAG (PO) 4- TAG(P - 1) 4- | 

The steps of Algorithm C are straightforward consequences of the storage 
layout ( 7 ); a slightly longer algorithm that is a little faster appears in exercise 15. 
In step C5, AVAIL is an abbreviation for LINK (LOC (AVAIL)), as shown in ( 9 ). 


C. The “buddy system.” We will now study another approach to dynamic 
storage allocation, suitable for use with binary computers. This method uses one 
bit of overhead in each block, and it requires all blocks to be of length 1 , 2 , 4 , 
8 , or 16, etc. If a block is not 2^ words long for some integer k , the next higher 
power of 2 is chosen and extra unused space is allocated accordingly. 

The idea of this method is to keep separate lists of available blocks of each 
size 2 , 0 < k < m. The entire pool of memory space under allocation consists 
of 2 m words, which can be assumed to have the addresses 0 through 2 m - 1 . 
Originally, the entire block of 2 m words is available. Later, when a block of 
2 k words is desired, and if nothing of this size is available, a larger available 
block is split into two equal parts; ultimately, a block of the right size 2 k will 
appear. When one block splits into two (each of which is half as large as the 
original), these two blocks are called buddies. Later when both buddies are 
available again, they coalesce back into a single block; thus the process can be 
maintained indefinitely, unless we run out of space at some point. 

The key fact underlying the practical usefulness of this method is that if we 
know the address of a block (the memory location of its first word), and if we 
also know the size of that block, we know the address of its buddy. For example, 
the buddy of the block of size 16 beginning in binary location 101110010110000 
is a block starting in binary location 101110010100000. To see why this must be 
true, we first observe that as the algorithm proceeds, the address of a block of 
size 2 is a multiple of 2 k . In other words, the address in binary notation has at 
least k zeros at the right. This observation is easily justified by induction: If it 
is true for all blocks of size 2 fc+1 , it is certainly true when such a block is halved. 

Therefore a block of size, say, 32 has an address of the form xx. . . xOOOOO 
(where the x’s represent either 0 or 1 ); if it is split, the newly formed buddy blocks 
have the addresses xx... xOOOOO and xx... xlOOOO. In general, let buddy* (a; ) = 
address of the buddy of the block of size 2 k whose address is x ; we find that 


buddy* (x) 


x + 2 fe , if x mod 2 k+1 — 0 ; 
x — 2 fc , if 2 : mod 2 k+1 — 2 k . 


( 10 ) 


This function is readily computed with the “exclusive or” instruction (sometimes 
called “selective complement” or “add without carry” ) usually found on binary 
computers; see exercise 28. 

The buddy system makes use of a one-bit TAG field in each block: 

TAG(P) = 0, if the block with address P is reserved; . 

TAG(P) = 1, if the block with address P is available. ( 1:L ' 
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This TAG field is present in all blocks, and it must not be tampered with by 
the users who reserve blocks. The available blocks also have two link fields, 
LINKF and LINKB, which are the usual forward and backward links of a doubly 
linked list; and they also have a KVAL field to specify k when their size is 2 k . 
The algorithms below make use of the table locations AVAIL [0] , AVAIL [1] , . . . , 
AVAIL [to] , which serve respectively as the heads of the lists of available storage 
of sizes 1, 2, 4, . . . , 2 m . These lists are doubly linked, so as usual the list heads 
contain two pointers (see Section 2.2.5): 

AVAILF [fc] = LINKF (LOC (AVAIL [fc] )) = link to rear of AVAIL [fc] list; , . 

AVAILB [fc] = LINKB (LOC (AVAIL [fc] ) ) = link to front of AVAIL [fc] list. 

Initially, before any storage has been allocated, we have 

AVAILF [to] = AVAILB [to] =0, 

LINKF (0) = LINKB(O) = LOC (AVAIL [m] ) , (13) 

TAG(O) = 1, KVAL(0)=to 

(indicating a single available block of length 2 m , beginning in location 0), and 

AVAILF [fc] = AVAILB [fc] = LOC (AVAIL [fc] ), for 0 < k < m (14) 

(indicating empty lists for available blocks of lengths 2 k for all k < m). 

From this description of the buddy system, the reader may find it enjoyable 
to design the necessary algorithms for reserving and freeing storage areas before 
looking at the algorithms given below. Notice the comparative ease with which 
blocks can be halved in the reservation algorithm. 

Algorithm R (Buddy system reservation). This algorithm finds and reserves 
a block of 2 k locations, or reports failure, using the organization of the buddy 
system as explained above. 

Rl. [Find block.] Let j be the smallest integer in the range k < j < m for 
which AVAILF [j] ^ LOC (AVAIL [j] ) , that is, for which the list of available 
blocks of size 2 3 is not empty. If no such j exists, the algorithm terminates 
unsuccessfully, since there are no known available blocks of sufficient size to 
meet the request. 

R2. [Remove from list.] Set L 4- AVAILB [j], P 4- LINKB (L), AVAILB [j] 4- P, 
LINKF (P) 4— LOC (AVAIL [j] ) , and TAG(L) 4-0. 

R3. [Split required?] If j = k, the algorithm terminates (we have found and 
reserved an available block starting at address L). 

R4. [Split.] Decrease j by 1. Then set P 4— L + 2 J , TAG(P) 4— 1, KVAL(P) 4— j, 
LINKF (P) 4- LINKB (P) 4— LOC (AVAIL [j] ) , AVAILF [j] 4— AVAILB [j] 4- P. 
(This splits a large block and enters the unused half in the AVAIL [j] list, 
which was empty.) Go back to step R3. | 

Algorithm S ( Buddy system liberation ). This algorithm returns a block of 2 k 
locations, starting in address L, to free storage, using the organization of the 
buddy system as explained above. 
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51. [Is buddy available?] Set P 4- buddy*. (L) . (See Eq. ( 10 ).) If k = m or if 
TAG(P) = 0, or if TAG(P) = 1 and KVAL(P) / k, go to S3. 

52. [Combine with buddy.] Set 

LINKF (LINKB (P)) ^LINKF(P), LINKB (LINKF(P) ) 4-LINKB(P). 

(This removes block P from the AVAIL [fc] list.) Then set k 4- k + 1, and if 
P < L set L 4— P. Return to SI. 

53. [Put on list.] Set TAG (L) 4- 1, P 4- AVAILF [A:] , LINKF(L) 4- P, LINKB (P) 4 -L. 
KVAL(L) 4 k, LINKB (L) 4— LDC (AVAIL [A] ) , AVAILF [A;] 4-L. (This puts 
block L on the AVAIL [A;] list.) | 

D. Comparison of the methods. The mathematical analysis of these dynamic 
storage-allocation algorithms has proved to be quite difficult, but there is one 
interesting phenomenon that is fairly easy to analyze, namely the “fifty-percent 
rule”: 

If Algorithms A and B are used continually in such a way that the system 
tends to an equilibrium condition, where there are N reserved blocks in the 
system, on the average, each equally likely to be the next one freed, and 
where the quantity K in Algorithm A takes on nonzero values (or, more 
generally, values > c as in step A4 1 ) with probability p, then the average 
number of available blocks tends to approximately \pN. 

This rule tells us approximately how long the AVAIL list will be. When the 
quantity p is near 1 — this will happen if c is very small and if the block sizes are 
infrequently equal to each other — we have about half as many available blocks 
as unavailable ones; hence the name “fifty-percent rule.” 

It is not hard to derive this rule. Consider the following memory map: 

A B C C B A B B B C B B 

I I I I I III I I I I II III II 

This shows the reserved blocks divided into three categories: 

A: when freed, the number of available blocks will decrease by one; 

B: when freed, the number of available blocks will not change; 

C: when freed, the number of available blocks will increase by one. 

Now let N be the number of reserved blocks, and let M be the number of 
available ones; let A, B, and C be the number of blocks of the types identified 
above. We have 

N = A + B + C 

M=\(2A + B + e) (l5 ^ 

where e = 0, 1, or 2 depending on conditions at the lower and upper boundaries. 

Let us assume that N is essentially constant, but that A, B, C, and e are 
random quantities that reach a stationary distribution after a block is freed and a 
(slightly different) stationary distribution after a block is allocated. The average 
change in M when a block is freed is the average value of (C -A)/N; the average 
change in M when a block is allocated is 1 — p. So the equilibrium assumption 
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tolls us that the average value of C — A — N + pN is zero. But then the average 
value of 2 M is pN plus the average value of e, since 2 M = N + A- C + e by ( 15 ). 
The fifty-percent rule follows. 

Our assumption that each deletion applies to a random reserved block will 
be valid if the lifetime of a block is an exponentially distributed random variable. 
On the other hand, if all blocks have roughly the same lifetime, this assumption 
is false; John E. Shore has pointed out that type A blocks tend to be “older” 
than type C blocks when allocations and liberations tend to have a somewhat 
first-in-first-out character, since a sequence of adjacent reserved blocks tends to 
be in order from youngest to oldest and since the most recently allocated block 
is almost never type A. This tends to produce a smaller number of available 
blocks, giving even better performance than the fifty-percent rule would predict. 
[See CACM 20 (1977), 812-820.] 

For more detailed information about the fifty-percent, rule, see D. J. M. 
Davies, BIT 20 (1980), 279-288; C. M. Reeves, Comp. J. 26 (1983), 25-35; 
G. Ch. Pflug, Comp. J. 27 (1984), 328-333. 

Besides this interesting rule, our knowledge of the performance of dynamic 
storage allocation algorithms is based almost entirely on Monte Carlo exper- 
iments. Readers will find it instructive to conduct their own simulation ex- 
periments when they are choosing between storage allocation algorithms for a 
particular machine and a particular application or class of applications. The 
author carried out several such experiments just before writing this section (and, 
indeed, the fifty-percent rule was noticed during those experiments before a 
proof for it was found); let us briefly examine the methods and results of those 
experiments here. 

The basic simulation program ran as follows, with TIME initially zero and 
with the memory area initially all available: 

PI. Advance TIME by 1. 

P2. Free all blocks in the system that are scheduled to be freed at the current 
value of TIME. 

P3. Calculate two quantities S (a random size) and T (a random lifetime), based 
on some probability distributions, using the methods of Chapter 3 . 

P 4. Reserve a new block of length S, which is due to be freed at (TIME + T). 
Return to PI. | 

Whenever TIME was a multiple of 200, detailed statistics about the performance 
of the reservation and liberation algorithms were printed. The same sequence 
of values of S and T was used for each pair of algorithms tested. After TIME 
advanced past 2000 , the system usually had reached a more or less steady state 
that gave every indication of being maintained indefinitely thereafter. However, 
depending on the total amount of storage available and on the distributions of 
S and T in step P3, the allocation algorithms would occasionally fail to find 
enough space and the simulation experiment was then terminated. 
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Let C be the total number of memory locations available, and let S and T 
denote the average values of S and T in step P3. It is easy to see that the expected 
number of unavailable words of memory at any given time is ST. once TIME 
is sufficiently large. When ST was greater than about |C in the experiments, 
memory overflow usually occurred, often before C words of memory were actually 
needed. The memory was able to become over 90 percent filled when the block 
size was small compared to C , but when the block sizes were allowed to exceed 
gC (as well as taking on much smaller values) the program tended to regard the 
memory as “full" when fewer than \C locations were actually in use. Empirical 
evidence suggests strongly that block sizes larger than should not be used 
with dynamic storage allocation if effective operation is expected. 

The reason for this behavior can be understood in terms of the fifty-percent 
rule: If the system reaches an equilibrium condition in which the size / of an 
average free block is less than the size r of an average block in use, we can expect 
to get an unfillable request unless a large free block is available for emergencies. 
Hence / > r in a saturated system that doesn’t overflow, and we have C — 
fM + rN > rM + rN ~ (p/2 + l)riV. The total memory in use is therefore 
tN < C / (p/2 + 1); when p rs 1 we are unable to use more than about 2/3 of the 
memory cells. 

The experiments were conducted with three size distributions for S: 


(51 ) an integer chosen uniformly between 100 and 2000; 

(52) sizes (1, 2, 4, 8, 16, 32) chosen with respective probabilities (~, |. 

16’ 32’ 32)’ 

(55) sizes (10, 12, 14, 16, 18, 20, 30, 40, 50, 60, 70, 80, 90, 100, 150, 200. 
250, 500, 1000, 2000, 3000, 4000) selected with equal probability. 

The time distribution T was usually a random integer chosen uniformly between 
1 and t. for fixed t = 10, 100, or 1000. 

Experiments were also made in which T was chosen uniformly between 1 and 
min( [I Hj, 1.2500) in step P3, where U is the number of time units remaining 
until the next scheduled freeing of some currently reserved block in the system. 
This time distribution was meant to simulate an “ almost- last-in-first-out” be- 


havioi. For if T were always chosen < U, the storage allocation system would 
degenerate into simply a stack operation requiring no complex algorithms. (See 
exercise 1.) The stated distribution causes T to be chosen greater than U about 
20 percent of the time, so we have almost, but not quite, a stack operation. When 
this distribution was used, algorithms such as A, B, and C behaved much better 
than usual; there were rarely, if ever, more than two items in the entire AVAIL 
list, while there were about 14 reserved blocks. On the other hand, the buddy 
system algorithms, R and S, were slower when this distribution was used, because 


they tend to split and coalesce blocks more frequently in a stack-like operation. 
The theoretical properties of this time distribution appear to be quite difficult 
to deduce (see exercise 32). 

Figure 42, which appeared near the beginning of this section, was the 
configuration of memory at TIME = 5000, with size distribution (SI) and with 
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Fig. 43. Memory map obtained with the best-fit method. (Compare this with Fig. 42, 
which shows the first-fit method, and Fig. 44, which shows the buddy system, for the 
same sequence of storage requests.) 


the times distributed uniformly in { 1 , , . . , 100}, using the first-fit method just as 
in Algorithms A and B above. For this experiment, the probability p that enters 
into the “fifty-percent rule” was essentially 1, so we would expect about half as 
many available blocks as reserved blocks. Actually Fig. 42 shows 21 available 
and 53 reserved. This does not disprove the fifty-percent rule: For example, 
at TIME = 4600 there were 25 available and 49 reserved. The configuration in 
Fig. 42 merely shows how the fifty-percent rule is subject to statistical variations. 
The number of available blocks generally ranged between 20 and 30, while the 
number of reserved blocks was generally between 45 and 55. 

Figure 43 shows the configuration of memory obtained with the same data 
as Fig. 1^2 but with the best-fit method used instead of the first-fit method. The 
constant c in step A4' was set to 16, to eliminate small blocks, and as a result 
the probability p dropped to about 0.7 and there were fewer available areas. 

When the time distribution was changed to vary from 1 to 1000 instead of 
1 to 100, situations precisely analogous to those shown in Figs. 42 and 43 were 
obtained, with all appropriate quantities approximately multiplied by 10. For 
example, there were 515 reserved blocks; and 240 free blocks in the equivalent 
of Fig. 42, 176 free blocks in the equivalent of Fig. 43. 

In all experiments comparing the best-fit, and first-fit methods, the latter 
always appeared to be superior. When memory size was exhausted, the first-fit 
method actually stayed in action longer than the best-fit method before memory 
overflow occurred, in most instances. 

The buddy system was also applied to the same data that led to Figs. 42 
and 43, and Fig. 44 was the result. Here, all sizes in the range 257 to 512 were 
treated as 512, those between 513 and 1024 were raised to 1024, etc. On the 
average this means that more than four thirds as much memory was requested 
(see exercise 21); the buddy system, of course, works better on size distributions 
like that of ( S2 ) above, instead of (SI). Notice that there are available blocks 
of sizes 2 9 , 2 10 , 2 11 , 2 12 , 2 13 , and 2 14 in Fig. 44. 

Simulation of the buddy system showed that it performs much better than 
might be expected. It is clear that the buddy system will sometimes allow two 
adjacent areas of the same size to be available without merging them into one 
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Fig. 44. Memory map obtained with the buddy system. (The tree structure indicates 
the division of certain large blocks into buddies of half the size. Squares indicate 
available blocks.) 



(if they are not buddies); but this situation is not present in Fig. 44 and, in 
fact, it is rare in piactice. In cases where memory overflow occurred, memory 
was 95 percent reserved, and this reflects a surprisingly good allocation balance. 
Furthermore, it was very seldom necessary to split blocks in Algorithm R., or to 
merge them m Algorithm S; the tree remained much like Fig. 44, with available 
blocks on the most commonly used levels. Some mathematical results that give 
insight into this behavior, at the lowest level of the tree, have been obtained by 
P. W. Purdom, Jr., and S. M. Stigler, JACM 17 (1970), 683-697. 

Another surprise was the excellent behavior of Algorithm A after the modifi- 
cation described in exercise 6; only 2.8 inspections of available block sizes were 
necessary on the average (using size distribution (SI ) and times chosen uniformly 
between 1 and 1000), and more than half of the time only the minimum value, 
one iteration, was necessary. This was true in spite of the fact that about 
250 available blocks were present. The same experiment with Algorithm A 
unmodified showed that about 125 iterations were necessary on the average 
(so about half of the AVAIL list was being examined each time); 200 or more 
iterations were found to be necessary about 20 percent of the time. 

This behavior of Algorithm A unmodified can, in fact, be predicted as a 
consequence of the fifty-percent rule. At equilibrium, the portion of memory 
containing the last half of the reserved blocks will also contain the last half of 
the free blocks; that portion will be involved half of the time when a block is 
freed, and so it must be involved in half of the allocations in order to maintain 
equilibrium. The same argument holds when one-half is replaced by any other 
fraction. (These observations are due to J. M. Robson.) 
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The exercises below include MIX programs for the two principal methods 
that are recommended as a consequence of the remarks above: (i) the boundary 
tag system, as modified in exercises 12 and 16; and (ii) the buddy system. Here 
are the approximate results: 

Time for reservation Time for liberation 

Boundary tag system: 33 + 7 A 18, 29, 31, or 34 

Buddy system: 19 + 25 R 27 + 26 S 

Here A > 1 is the number of iterations necessary when searching for an available 
block that is large enough; R > 0 is the number of times a block is split in two 
(the initial difference of j — k in Algorithm R); and S' > 0 is the number of times 
buddy blocks are reunited during Algorithm S. The simulation experiments 
indicate that under the stated assumptions with size distribution (SI) and time 
chosen between 1 and 1000, we may take A = 2.8, R = S = 0.04 on the average. 
(The average values A = 1.3, R ~ S = 0.9 were observed when the “almost-last- 
in-first-out” time distribution was substituted as explained above.) This shows 
that both methods are quite fast, with the buddy system slightly faster in Mix’s 
case. Remember that the buddy system requires about 44 percent more space 
when block sizes are not constrained to be powers of 2. 

A corresponding time estimate for the garbage collection and compacting 
algorithm of exercise 33 is about 104 units of time to locate a free node, assuming 
that garbage collection occurs when the memory is approximately half full, and 
assuming that the nodes have an average length of 5 words with 2 links per 
node. The pros and cons of garbage collection are discussed in Section 2.3.5. 
When the memory is not heavily loaded and when the appropriate restrictions 
are met, garbage collection and compacting is very efficient; for example, on the 
MIX computer, the garbage collection method is faster than the other two, if the 
accessible items never occupy more than about one-third of the total memory 
space, and if the nodes are relatively small. 

If the assumptions underlying garbage collection are met, the best strategy 
may be to divide the pool of memory into two halves and to do all allocation 
sequentially within one half. Instead of freeing blocks as they become available, 
we simply wait until the current active half of memory is full; then we can copy all 
active data to the other half, simultaneously removing all holes between blocks, 
with a method like that of exercise 33. The size of each half pool might also be 
adjusted as we switch from one half to the other. 

The simulation techniques mentioned above were applied also to some other 
storage allocation algorithms. The other methods were so poor by comparison 
with the algorithms of this section that they will be given only brief mention here: 

a) Separate AVAIL lists were kept for each size. A single free block was 
occasionally split into two smaller blocks when necessary, but no attempt was 
made to put such blocks together again. The memory map became fragmented 
into hirer and finer parts until it was in terrible shape; a simple scheme like this 
is almost equivalent to doing separate allocation in disjoint areas, one area for 
each block size. 
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b) An attempt was made to do two-level allocation: The memory was divided 
into 32 large sectors. A brute-force allocation method was used to reserve large 
blocks of 1, 2, or 3 (rarely more) adjacent sectors; each large block such as this 
was subdivided to meet storage requests until no more room was left within 
the current large block, and then another large block was reserved for use in 
subsequent allocations. Each large block was returned to free storage only when 
all space within it became available. This method almost always ran out of 
storage space very quickly. 

Although this particular method of two-level allocation was a failure for 
the data considered in the author’s simulation experiments, there are other 
circumstances (which occur not infrequently in practice) when a multiple-level 
allocation strategy can be beneficial. For example, if a rather large program 
operates in several stages, we might know that certain types of nodes are needed 
only within a certain subroutine. Some programs might also find it desirable 
to use quite different allocation strategies for different classes of nodes. The 
idea of allocating storage by zones, with possibly different strategies employed 
in each zone and with the ability to free an entire zone at once, is discussed by 
Douglas T. Ross in CACM 10 (1967), 481 492. 

For further empirical results about dynamic storage allocation, see the arti- 
cles by B. Randell, CACM 12 (1969), 365-369, 372; R W. Purdom, S. M. Stigler. 
and T. O. Cheam, BIT 11 (1971), 187 195; B. H. Margolin, R. P. Parmelee, and 
M. Schatzoff, IBM Systems J. 10 (1971), 283-304; J. A. Campbell, Comp. J. 
14 (1971), 7-9; John E. Shore, CACM 18 (1975), 433-440; Norman R. Nielsen, 
CACM 20 (1977), 864-873. 

*E. Distributed fit. If the distribution of block sizes is known in advance, 
and if each block present is equally likely to be the next one freed regardless 
of when it was allocated, we can use a technique that has substantially better 
memory utilization than the general-purpose techniques described so far, by 
following the suggestions of E. G. Coffman, Jr., and F. T. Leighton [J. Computer 
and System Sci. 38 (1989), 2-35];, Their “distributed-fit method” works by 
partitioning memory into roughly N + VN lg N slots, where N is the desired 
maximum number of blocks to be handled in steady state. Each slot has a fixed 
size, although different slots may have different sizes; the main point is that any 
given slot has fixed boundaries, and it will either be empty or contain a single 
allocated block. 

The first N slots in Coffman and Leighton’s scheme are laid out according 
to the assumed distribution of sizes, while the last VN lg N slots all have the 
maximum size. For example, if we assume that the block sizes will be uniformly 
distributed between 1 and 256, and if we expect to handle N = 2 14 such blocks, 
we would divide the memory into jV/256 = 2 6 slots of each size 1, 2, . . . , 256, 
followed by an “overflow area” that contains VN lg N - 2 7 • 14 = 1792 blocks of 
size 256. When the system is operating at full capacity, we expect it to handle 
N blocks of average size ^|Z, occupying ^-N = 2 21 + 2 13 = 2,105,344 locations; 
this is the amount of space we have allocated to the first N slots. We have also 
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set aside an additional 1792 • 256 = 458,752 locations to handle the effects of 
random variations; this additional overhead amounts to 0(N~ 1 ^ 2 log N) of the 
total space, rather than a constant multiple of N as in the buddy system, so it 
becomes a negligible fraction when N -A oc. In our example, however, it still 
amounts to about 18% of the total allocation. 

The slots should be arranged in order so that the smaller slots precede the 
larger ones. Given this arrangement, we can allocate blocks by using either the 
first-fit or the best-fit technique. (Both methods are equivalent in this case, 
because the slot sizes are ordered.) The effect, under our assumptions, is to 
start searching at an essentially random place among the first N slots whenever 
a new allocation request comes in, and to continue until we find an empty slot. 

If the starting slot for each search is truly random between 1 and N, we will 
not have to invade the overflow area very often. Indeed, if we insert exactly N 
items starting at random slots, overflow will occur only 0(\fN) times, on the 
average. The reason is that we can compare this algorithm to hashing with linear 
probing (Algorithm 6.4L), which has the same behavior except that the search 
for an empty cell wraps around from N to 1 instead of going into an overflow 
area. The analysis of Algorithm 6.4L in Theorem 6.4K shows that, when N 
items have been inserted, the average displacement of each item from its hash 
address is |(Q(iV) — 1) ~ y/ nN/8 ; by circular symmetry this average is easily 
seen to be the same as the average number of times a search goes from slot k 
to slot k + 1, for each k. Overflows in the distributed-fit method correspond to 
searches that go from slot N to slot 1, except that our situation is even better 
because we avoid some congestion by not wrapping around. Therefore fewer than 
V nN/8 overflows will occur, on the average. This analysis does not take account 
of deletions, which preserve the assumptions of Algorithm 6.4L only if we move 
blocks back when deleting another block that intervened between their starting 
slots and their allocated slots (see Algorithm C.4R); again, however, moving 
them back would only increase the chance of overflow. Our analysis also fails to 
account for the effect of having more than N blocks present at once; this can 
happen if we assume only that the arrival time between blocks is about one Nth 
of the residence time. For the case of more than N blocks we need to extend the 
analysis of Algorithm 6.4L, but Coffman and Leighton proved that the overflow 
area will almost never need more than VN lg N slots; the probability of running 
off the end is less than 0{N~ M ) for all M. 

In our example, the starting slot for the search during an allocation is not 
uniform among slots 1, 2, . . . , IV; it is, instead, uniform among slots 1, 65, 129, 
. . . , N — 63, because there are A r /256 = 64 slots of each size. But this deviation 
from the random model considered in the previous paragraph makes overflow 
even less likely than predicted. All bets are off, of course, if the assumptions 
about block size distribution and occupancy time are violated. 

F . Overflow. What do we do when no more room is available? Suppose there 
is a request for, say, n consecutive words, when all available blocks are too small. 
The first time this happens, there usually are more than n available locations 
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present, but they are not consecutive; compacting memory (that is, moving 
some of the locations that are in use, so that all available locations are brought 
together) would mean that we could continue processing. But compacting is slow, 
and it requires a disciplined use of pointers; moreover, the vast majority of cases 
in which the first-fit method runs out of room will soon thereafter run completely 
out of space anyway, no matter how much compacting and re-compacting is 
done. Therefore it is generally not worthwhile to write a compacting program, 
except under special circumstances in connection with garbage collection, as 
in exercise 33. If overflow is expected to occur, some method for removing 
items from memory and storing them on an external memory device can be 
used, with provision for bringing the information back again when it is needed. 
This implies that all programs referring to the dynamic memory area must be 
severely restricted with regard to the allowable references they make to other 
blocks, and special computer hardware (for example, interrupt on absence of 
data, or automatic “paging”) is generally required for efficient operation under 
these conditions. 

Some decision procedure is necessary to decide which blocks are the most 
likely candidates for removal. One idea is to maintain a doubly linked list of the 
reserved blocks, in which a block is moved up to the front of the list each time 
it is accessed; then the blocks are effectively sorted in order of their last access, 
and the block at the rear of the list is the one to remove first. A similar effect 
can be achieved more simply by putting the reserved blocks into a circular list 
and including a “recently used” bit in each block; the latter is set to 1 whenever 
the block is accessed. When it is time to remove a block, a pointer moves along 
the circular list, resetting all “recently used” bits to 0 until finding a block that 
has not been used since the last time the pointer reached this part of the circle. 

J. M. Robson has shown [JACM 18 (1971), 416-423] that dynamic storage 
allocation strategies that never relocate reserved blocks cannot possibly be guar- 
anteed to use memory efficiently; there will always be pathological circumstances 
in which the method breaks down. For example, even when blocks are restricted 
to be of sizes 1 and 2, overflow might occur with the memory only about ^ full, 
no matter what allocation algorithm is used! Robson’s interesting results are 
surveyed in exercises 36—40, and in exercises 42—43 where he has shown that the 
best-fit method has a very bad worst case by comparison with first-fit. 

G. For further reading. A comprehensive survey and critical review of 
dynamic storage allocation techniques, based on many more years of experience 
than were available to the author when the material above was written, has 
been compiled by Paul R. Wilson, Mark S. Johnstone, Michael Neely, and David 
Boles, Lecture Notes in Computer Science 986 (1995), 1-116. 

EXERCISES 

1. [20] What simplifications can be made to the reservation and liberation algo- 
rithms of this section, if storage requests always appear in a “last-in-first-out” manner, 
that is, if no reserved block is freed until after all blocks that were reserved subsequently 
have already been freed? 
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2. [HM23] (E. Wolman.) Suppose that we want to choose a fixed node size for 
variable-length items, and suppose also that when each node has length k and when an 
item has length l, it takes \l/(k — b)"| nodes to store this item. (Here b is a constant, 
signifying that b words of each node contain control information, such as a link to 
the next node.) If the average length l of an item is L, what choice of k minimizes 
the average amount of storage space required? (Assume that the average value of 
( l/(k — b )) mod 1 is equal to 1/2, for any fixed k, as l varies.) 

3. [40] By computer simulation, compare the best-fit, first-fit, and worst-fit methods 
of storage allocation; in the latter method, the largest available block is always chosen. 
Is there any significant difference in the memory usage? 

4. [22] Write a MIX program for Algorithm A, paying special attention to making 
the inner loop fast. Assume that the SIZE field is (4:5), the LINK field is (0:2), and 
A < 0. 

► 5. [1<S] Suppose it is known that N is always 100 or more in Algorithm A. Would it 
be a good idea to set c = 100 in the modified step A4'? 

► 6. [23] (Next fit.) After Algorithm A has been used repeatedly, there will be a strong 
tendency for blocks of small SIZE to remain at the front of the AVAIL list, so that it will 
often be necessary to search quite far into the list before finding a block of length N or 
more. For example, notice how the size of the blocks essentially increases in Fig. 42, 
for both reserved and free blocks, from the beginning of memory to the end. (The 
AVAIL list used while Fig. 42 was being prepared was kept sorted by order of location, 
as required by Algorithm B.) Can you suggest a way to modify Algorithm A so that 
(a) short blocks won’t tend to accumulate in a particular area, and (b) the AVAIL list 
may still be kept in order of increasing memory locations, for purposes of algorithms 
like Algorithm B? 

7. [10] The example (l) shows that first- fit can sometimes be definitely superior to 
best-fit. Give a similar example that shows a case where best-fit. is superior to first-fit. 

8. [21] Show how to modify Algorithm A in a simple way to obtain an algorithm for 
the best-fit method, instead of first-fit. 

► 9. [26] In what ways could a reservation algorithm be designed to use the best-fit 
method, without searching through the whole AVAIL list? (Try to think of ways that 
cut down the necessary search as much as possible.) 

10. [22] Show how to modify Algorithm B so that the block of N consecutive cells 
beginning at location P0 is made available, without assuming that each of these N cells 
is currently unavailable; assume, in fact, that the area being freed may actually overlap 
several blocks that are already free. 

11. [ M25 ] Show that the improvement to Algorithm A suggested in the answer to 
exercise 6 can also be used to lead to a slight improvement in Algorithm B, which cuts 
the average length of search from half the length of the AVAIL list to one-third this 
length. (Assume that the block being freed will be inserted into a random place within 
the sorted AVAIL list.) 

► 12. [20] Modify Algorithm A so that it follows the boundary-tag conventions of 
(t) — (9 ) 7 uses the modified step A4' described in the text, and also incorporates the 
improvement of exercise 6. 

13. [21] Write a MIX program for the algorithm of exercise 12. 
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14. [21] What difference would it make to Algorithm C and the algorithm of exer- 
cise 12, (a) if the SIZE field were not present in the last word of a free block? or (b) if 
the SIZE field were not present in the first word of a reserved block? 

► 15. [ 24 ] Show how to speed up Algorithm C at the expense of a slightly longer 
program, by not changing any more links than absolutely necessary in each of four 
cases depending on whether TAG(PO - 1). TAG(P0 + SIZE(PO)) are plus or minus. 

16. [24] Write a MIX program for Algorithm C, incorporating the ideas of exercise 15. 

17. [10] What should the contents of LOC(AVAIL) and LOC(AVAIL) + 1 be in (9) when 
there are no available blocks present? 

► 18. [20] Figures 42 and 43 were obtained using the same data, and essentially the 
same algorithms (Algorithms A and B), except that Fig. 43 was prepared by modifying 
Algorithm A to choose best-fit instead of first-fit. Why did this cause Fig. 42 to have a 
large available area in the higher locations of memory, while in Fig. 43 there is a large 
available area in the lower locations? 

► 19. [24] Suppose that blocks of memory have the form of (7), but without the TAG or 
SIZE fields required in the last word of the block. Suppose further that the following 
simple algorithm is being used to make a reserved block free again: Q •<— AVAIL. 
LINK(PO) <— Q. LINK(P0 + 1) LOC (AVAIL) . LINK(Q + 1) <- PO, AVAIL «— PO. TAG(PO) •<— 

— . (1 his algorithm does nothing about collapsing adjacent areas together.) 

Design a reservation algorithm, similar to Algorithm A, that does the necessary 
collapsing of adjacent free blocks while searching the AVAIL list, and at the same time 
avoids any unnecessary fragmentation of memory as in (2), (3), and (4). 

20. [00] Why is it desirable to have the AVAIL [fc] lists in the buddy system doubly 
linked instead of simply having straight linear lists? 

21. [HM25] Examine the ratio a n /b n , where a n is the sum of the first n terms of 
1+2 + 4 + 4 + 8 + 8 + 8 + 8 + 16+ 16 + •••, and b n is the sum of the first n terms of 
l + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 +---,asn goes to infinity. 

► 22. [21] The text repeatedly states that the buddy system allows only blocks of size 2 k 
to be used, and exercise 21 shows this can lead to a substantial increase in the storage 
required. But if an 11-word block is needed in connection with the buddy system, why 
couldn t we find a 16-word block and divide it into an 11- word piece together with two 
free blocks of sizes 4 and 1? 

23. [05] What is the binary address of the buddy of the block of size 4 whose binary 
address is 011011110000? What would it be if the block were of size 16 instead of 4? 

24. [20] According to the algorithm in the text, the largest block (of size 2 m ) has no 
buddy, since it represents all of storage. Would it be correct to define buddy m (0) = 0 
(namely, to make this block its own buddy), and then to avoid testing k = m instep SI? 

► 25. [22] Criticize the following idea: “Dynamic storage allocation using the buddy 
system will never reserve a block of size 2 m in practical situations (since this would fill 
the whole memory), and, in general, there is a maximum size 2 n for which no blocks of 
greater size will ever be reserved. Therefore it is a waste of time to start with such large 
blocks available, and to combine buddies in Algorithm S when the combined block has 
a size larger than 2".” 

► 26. [21] Explain how the buddy system could be used for dynamic storage allocation 
in memory locations 0 through M - 1 even when M does not have the form 2 m as required 
in the text. 
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27. [24] Write a MIX program for Algorithm R, and determine its running time. 

28. [25] Assume that MIX is a binary computer, with a new operation code XOR defined 
as follows (using the notation of Section 1.3.1): “C = 5, F = 5. For each bit position in 
location M that equals 1, the corresponding bit position in register A is complemented 
(changed from 0 to 1 or 1 to 0); the sign of rA is unaffected. The execution time is 2u.” 

Write a MIX program for Algorithm S, and determine its running time. 

29. [20] Could the buddy system do without the tag bit in each reserved block? 

30. [M 48 ] Analyze the average behavior of Algorithms R and S, given reasonable 
distributions for the sequence of storage requests. 

31. [M 40 ] Can a storage allocation system analogous to the buddy system be designed 
using the Fibonacci sequence instead of powers of two? (Thus, we might start with F m 
available words, and split an available block of F k words into two buddies of respective 
lengths Ffr_i and F k - 2 -) 

32. [HM 46 ] Determine lim n _ >00 a n , if it exists, where a n is the mean value of t n in 
a random sequence defined as follows: Given the values of t k for 0 < k < n, let t n be 
chosen uniformly from {1, 2, . . . ,<?„}, where 

g n = [_ | min(l0000, /( f„_i - 1), /(f„_ 2 - 2), ... , f(t 0 - n))J, 

and f(x) x if x > 0, f(x) = 00 if x < 0. [Note: Some limited empirical tests indicate 
that a„ might be approximately 14, but this is probably not very accurate.] 

► 33. [28] ( Garbage collection and compacting.) Assume that memory locations 1, 2, 
. . . , AVAIL — 1 are being used as a storage pool for nodes of varying sizes, having the 
following form: The first word of NODE (P) contains the fields 


SIZE(P) = number of words in NODE(P); 

T (P) = number of link fields in NODE(P): T(P) < SIZE(P): 

LINK(P) = special link field for use only during garbage collection. 

The node immediately following NODE(P) in memory is N0DE(P + SIZE(P) ) . Assume 
that the only fields in NODE(P) that are used as links to other nodes are LINK(P + 1), 
LINK(P + 2), . . . , LINK(P + T(P)), and each of these link fields is either A or the address 
of the first word of another node. Finally, assume that there is one further link variable 
in the program, called USE, and it points to one of the nodes. 

Design an algorithm that (i) determines all nodes accessible directly or indirectly 
from the variable USE, (ii) moves these nodes into memory locations 1 through K — 1, for 
some K, changing all links so that structural relationships are preserved, and (iii) sets 
AVAIL «— K. 

For example, consider the following contents of memory, where INFO(L) denotes 
the contents of location L, excluding LINK(L): 


1: SIZE = 2, T = 1 6: SIZE = 2, T = 0 

2: LINK = 6, INFO = A 7: CONTENTS = D 

3: SIZE = 3, T = 1 8: SIZE = 3, T = 2 

4: LINK = 8, INFO = B 9: LINK = 8, INFO = E 

5: CONTENTS = C 10: LINK = 3, INFO = F 

Your algorithm should transform this into 

1: SIZE = 3, T = 1 4: SIZE = 3, T = 2 

2: LINK = 4, INFO = B 5: LINK = 4. INFO = E 

3: CONTENTS = C 6: LINK = 1, INFO = F 


AVAIL = 11, 
USE = 3. 


AVAIL = 7, 
USE = 1. 
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34. [29] Write a MIX program for the algorithm of exercise 33, and determine its 
running time. 

35. [22] Contrast the dynamic storage allocation methods of this section with the 
techniques for variable-size sequential lists discussed at the end of Section 2.2.2. 

► 36. [20] A certain lunch counter in Hollywood, California, contains 23 seats in a row. 
Diners enter the shop in groups of one or two, and a glamorous hostess shows them 
where to sit. Prove that she will always be able to seat people immediately without 
splitting up any pairs, if no customer who comes alone is assigned to any of the seats 
numbered 2, 5, 8, ... , 20, provided that there never are more than 16 customers present 
at a time. (Pairs leave together.) 

► 37. [26] Continuing exercise 36, prove that the hostess can’t always do such a good 
job when there are only 22 seats at the counter: No matter what strategy she uses, 
it will be possible to reach a situation where two friends enter and only 14 people are 
seated, but no two adjacent seats are vacant. 

38. [M21] (J. M. Robson.) The lunch-counter problem in exercises 36 and 37 can be 
generalized to establish the worst-case performance of any dynamic storage allocation 
algorithm that never relocates reserved blocks. Let N(n, to) be the smallest amount of 
memory such that any series of requests for allocation and liberation can be handled 
without overflow, provided that all block sizes are < to and the total amount of space 
requested never exceeds n. Exercises 36 and 37 prove that JV(16,2) = 23: determine 
the exact value of N(n, 2) for all n. 

39. [HM23] (J. M. Robson.) In the notation of exercise 38, show that N(n i +n 2 , to) < 
IV(m, m) + N(n 2 , to) + N(2m — 2, to); hence for fixed to, lim™-^ N{n, m)/n = N(m ) 
exists. 

40. [HM50] Continuing exercise 39, determine JV(3), JV(4), and lim m ^ 00 iV(m)/lgm 
if it exists. 

41. [M27] The purpose of this exercise is to consider the worst-case memory usage 
of the buddy system. A particularly bad case occurs, for example, if we start with an 
empty memory and proceed as follows: First reserve n = 2 r+1 blocks of length 1, which 
go into locations 0 through n - 1; then for k = 1, 2, . . . , r, liberate all blocks’ whose 
starting location is not divisible by 2 k , and reserve 2 ~ k ~ 1 n blocks of length 2 k , which 
go into locations 4(1 + k)n through ±(2 + k)n - 1. This procedure uses 1 + \r times 
as much memory as is ever occupied. 

Prove that the worst case cannot be substantially worse than this: When all 
requests are for block sizes 1, 2, . . . , 2 r , and if the total space requested at any time 
never exceeds n, where n is a multiple of 2 r , the buddy system will never overflow a 
memory area of size (r + l)n. 

42. [MJfO] (J. M. Robson, 1975.) Let NBF(n,m) be the amount of memory needed to 
guarantee non-overflow when the best-fit method is used for allocation as in exercise 38. 
Find an attacking strategy to show that N BF (n, to) > mn - 0(n + to 2 ). 

43. [HM35] Continuing exercise 42, let N FF (n,m) be the memory needed when the 
first-fit method is used. Find a defensive strategy to show that N FF (n , to) < H m n/ In 2. 
(Hence the worst case of first-fit is not far from the best possible worst case.) 

44. [M21] Suppose the distribution function F{x) = (probability that a block has 
size < x) is continuous. For example, F(x) is (x - a)/(b - a) for a < x < b if the sizes 
are uniformly distributed between a and b. Give a formula that expresses the sizes of 
the first N slots that should be set up when we use the distributed-fit method. 
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2.6. HISTORY AND BIBLIOGRAPHY 

Linear lists and rectangular arrays of information kept in consecutive memory 
locations were widely used from the earliest days of stored-program computers, 
and the earliest treatises on programming gave the basic algorithms for traversing 
these structures. [For example, see J. von Neumann, Collected Works 5, 113- 
116 (written 1946); M. V. Wilkes, D. J. Wheeler, S. Gill, The Preparation 
of Programs for an Electronic Digital Computer (Reading, Mass.: Addison- 
Wesley, 1951), subroutine V-l; and see especially also the work of Konrad Zuse, 
Berichte der Gesellschaft fiir Mathematik und Datenverarbeitung 63 (Bonn: 
1972), written in 1945. Zuse was the first to develop nontrivial algorithms 
that worked with lists of dynamically varying lengths.] Before the days of 
index registers, operations on sequential linear lists were done by performing 
arithmetic on the machine language instructions themselves, and the need to do 
such arithmetic was one of the early motivations for having a computer whose 
programs share memory space with the data they manipulate. 

Techniques that permit variable-length linear lists to share sequential loca- 
tions, in such a way that they shift back and forth when necessary as described 
in Section 2.2.2, were apparently a much later invention. J. Dunlap of Digitek 
Corporation developed such techniques before 1963 in connection with the de- 
sign of a series of compiler programs; about the same time the idea appeared 
independently in the design of a COBOL compiler at IBM Corporation, and a 
collection of related subroutines called CITRUS was subsequently used at various 
installations. The techniques remained unpublished until after they had been 
developed independently by Jan Garwick of Norway; see BIT 4 (1964), 137-140. 

The idea of having linear lists in nonsequential locations seems to have orig- 
inated in connection with the design of computers with rotating drum memories. 
After executing the instruction in location n, such a computer was usually not 
ready to get its next instruction from location n + 1, because the drum had 
already rotated past that point. Depending on the instruction being performed, 
the most favorable position for the next instruction might be n + 7 or n + 18, 
say, and the machine could operate up to six or seven times faster if its in- 
structions were located optimally rather than consecutively. [For a discussion of 
the interesting problems concerning the best placement of instructions, see the 
author’s article in JACM 8 (1961), 119-150.] Therefore an extra address field 
was provided in each machine language instruction, to serve as a link to the next 
command. This idea, called “one-plus-one addressing,” was discussed by John 
Mauchly in 1946 [Theory and Techniques for the Design of Electronic Comput- 
ers 4 (U. of Pennsylvania, 1946), Lecture 37]; it contained the notion of linked 
lists in embryonic form, although the dynamic insertion and deletion operations 
that we have used so frequently in this chapter were still unknown. Another 
early appearance of links in programs was in H. P. Luhn’s 1953 memorandum 
suggesting the use of “chaining” for external searching; see Section 6.4. 

Linked memory techniques were really born when A. Newell, J. C. Shaw, 
and H. A. Simon began their investigations of heuristic problem-solving by 
machine. As an aid to writing programs that searched for proofs in mathematical 
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logic, they designed the first List-processing language, IPL-II, in the spring of 
1956. (IPL was an acronym for Information Processing Language.) This was 
a system that made use of pointers and included important concepts like the 
list of available space, but the concept of stacks was not yet well developed. 
IPL-III, designed a year later, included “push down and “pop up” for stacks as 
important basic operations. [For references to IPL-II see IRE Transactions IT-2 
(September 1956), 61-70; Proc. Western Joint Comp. Conf. 9 (1957), 218-240. 
Material on IPL-III first appeared in course notes given at the University of 
Michigan in the summer of 1957.] 

The work of Newell, Shaw, and Simon inspired many other people to use 
linked memory, which was often referred to as NSS memory at the time, but 
mostly for problems dealing with simulation of human thought processes. Grad- 
ually, the techniques became recognized as basic computer-programming tools; 
the first article describing the usefulness of linked memory for “down-to-earth” 
problems was published by J. W. Carr, III. in CACM 2,2 (February 1959), 4-6. 
Cari pointed out in this article that linked lists can readily be manipulated in 
ordinary programming languages, without requiring sophisticated subroutines 
or interpretive systems. See also G. A. Blaauw, “Indexing and control-word 
techniques,” IBM J. Res. and Dev. 3 (1959), 288-301. 

At first, one-word nodes were used for linked tables, but about 1959 the 
usefulness of several consecutive words per node and “multilinked” lists was 
gradually being discovered by several different groups of people. The first article 
dealing specifically with this idea was published by D. T. Ross, CACM 4 (1961), 
147-150. At that time he used the term “plex” for what has been called a “node” 
in this chapter, but he subsequently used the word “plex in a different sense to 
denote a class of nodes combined with associated algorithms for their traversal. 

Notations for referring to fields within nodes are generally of two kinds: The 
name of the field either precedes or follows the pointer designation. Thus, while 
we have written “INFO(P)” in this chapter, some other authors write, for exam- 
ple, “P . INFO" . At the time this chapter was prepared, the two notations seemed 
to be equally prominent. The notation adopted here has the great advantage 
that it translates immediately into FORTRAN, COBOL, or similar languages, if 
we define INFO and LINK arrays and use P as the index. Furthermore it seems 
natural to use mathematical functional notation to describe attributes of a node. 
Note that “INFO(P)” is pronounced “info of P” in conventional mathematical 
verbalization, just as f(x) is rendered “/ of x.” The alternative notation P . INFO 
has less of a natural flavor, since it tends to put the emphasis on P, although it 
can be read P s info ; the reason INFO(P) seems preferable is apparently the 
fact that P is variable, but INFO has a fixed significance when the notation is 
employed. By analogy, we could consider a vector A = (A[l], A[2], . . . , A[100]) 

to be a node having 100 fields named 1,2, 100. Now the second field would 

be referred to as “2(P)” in our notation, where P points to the vector A; but 
if we are referring to the jth element of the vector, we find it more natural to 
write A[j], putting the variable quantity “j” second. Similarly it seems most 
appropriate to put the variable quantity “P” second in the notation INF0(P). 


2.6 


HISTORY AND BIBLIOGRAPHY 


459 


Perhaps the first people to recognize that the concepts “stack” (last-in- first- 
out) and "queue’ (first-in-first-out) are important objects of study were cost 
accountants interested in reducing income tax assessments; for a discussion of 
the “LIFO' and “FIFO” methods of pricing inventories, see any intermediate ac- 
counting textbook, e.g., C. F. and W. ,T. Schlatter, Cost Accounting (New York: 
Wiley, 1957), Chapter 7. In the mid-1940s, A. M. Turing developed a stack mech- 
anism called Reversion Storage for subroutine linkage, local variables, and pa- 
rameters. His names for “push” and “pop” were “bury” and “disinter/unbury.” 
(See the references in Section 1.4.5.) No doubt simple uses of stacks kept in 
sequential memory locations were common in computer programming from the 
earliest days, since a stack is such an intuitive concept. The programming of 
stacks in linked form appeared first in IPL, as stated above; the name “stack” 
stems from IPL terminology (although “pushdown list” was the more official IPL 
wording), and it was also independently introduced by E. W. Dijkstra [Numer. 
Math. 2 (I960), 312-318], “Deque” is a term coined by E. J. Schweppe in 1966. 

The origin of circular and doubly linked lists is obscure; presumably these 
ideas occurred naturally to many people. A strong factor in the popularization 
of such techniques was the existence of general List-processing systems based on 
them [principally the Knotted List Structures, CACM 5 (1962), 161 165, and 
Symmetric List Processor, CACM 6 (1963), 524—544, of J. Weizenbaum] . Ivan 
Sutherland introduced the use of independent doubly linked lists within larger 
nodes, in his Sketchpad system (Ph.D. thesis, Mass. Inst, of Technology, 1963). 

Various methods for addressing and traversing multidimensional arrays of 
information were developed independently by clever programmers since the ear- 
liest days of computers, and thus another part of the unpublished computer 
folklore was born. This subject was first surveyed in print by H. Hellerman, 
CACM 5 (1962), 205-207. See also J. C. Gower, Comp. J. 4 (1962), 280-286. 

Tree structures represented explicitly in computer memory were originally 
used for applications to algebraic formula manipulation. The machine language 
for several early computers used a three-address code to represent the compu- 
tation of arithmetic expressions; the latter is equivalent to the INFO, LLINK, 
and RLINK of a binary tree representation. In 1952, H. G. Kahrimanian devel- 
oped algorithms for differentiating algebraic formulas represented in an extended 
three-address code; see Symposium on Automatic Programming (Washington, 
D.C.: Office of Naval Research, May 1954), 6-14. 

Since then, tree structures in various guises have been studied independently 
by many people in connection with numerous computer applications, but the 
basic techniques for tree manipulation (not general List manipulation) have 
seldom appeared in print except in detailed description of particular algorithms. 
The first general survey was made in connection with a more general study of 
all data structures by K. E. Iverson and L. R. Johnson [IBM Corp. research 
reports RC-390, RC-603, 1961; see Iverson, A Programming Language (New 
York: Wiley, 1962), Chapter 3]. See also G. Salton, CACM 5 (1962), 103-114. 

The concept of threaded trees is due to A. J. Perlis and C. Thornton, CACM 
3 (I960), 195-204. Their paper also introduced the important idea of traversing 
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trees in various orders, and gave numerous examples of algebraic manipulation 
algorithms. Unfortunately, this important paper was prepared hastily and it 
contains many misprints. The threaded lists of Perlis and Thornton were only 
•‘right-threaded trees” in our terminology; binary trees that are threaded in 
both directions were independently discovered by A. W. Holt, A Mathematical 
and Applied Investigation of Tree Structures (Thesis, U. of Pennsylvania, 1963). 
Postorder and preorder for the nodes of trees were called “normal along order” 
and “dual along order” by Z. Pawlak [Colloquium on the Foundation of Math- 
ematics, Tihany, 1962 (Budapest; Akademiai Kiado, 1965), 227 238], Preorder 
was called “subtree order” by Iverson and Johnson in the references cited above. 
Graphical ways to represent the connection between tree structures and cor- 
responding linear notations were described by A. G. Oet.tinger, Proc. Harvard 
Symp. on Digital Computers and their Applications (April 1961), 203 224. The 
representation of trees in preorder by degrees, with associated algorithms relating 
this representation to Dewey decimal notation and other properties of trees, was 
presented by S. Corn, Proc. Symp. Math. Theory of Automata (Brooklyn: Poly. 
Inst., 1962), 223-240. 

The history of tree structures as mathematical entities, together with a 
bibliography of the subject, is reviewed in Section 2. 3. 4. 6. 

At the time this section was first written in 1966, the most widespread 
knowledge about information structures was due to programmers’ exposure to 
List processing systems, which played a very important part in this history. 
The first widely used system was IPL-V (a descendant of IPL-III, developed late 
in 1959); IPL-V was an interpretive system in which a programmer learned a 
machine-like language for List operations. At about the same time, FLPL (a set 
of FORTRAN subroutines for List manipulation, also inspired by IPL but using 
subroutine calls instead of interpretive language) was developed by H. Gelernter 
and others. A third system, LISP, was designed by J. McCarthy, also in 1959. 
LISP was quite different from its predecessors: Its programs were (and still 
are) expressed in mathematical functional notation combined with “conditional 
expressions,” then converted into a List representation. Many List processing 
systems came into existence during the 1960s; the most prominent among these 
from a historical standpoint was J. Weizenbaum’s SLIP, a set. of subroutines that 
implemented doubly linked Lists in FORTRAN. 

An article by Bobrow and Raphael, CACM 7 (1964), 231-240, may be 
read as a brief introduction to IPL-V, LISP, and SLIP; it gives a comparison 
of these systems. An excellent early introduction to LISP was published by 
P. M. Woodward and D. P. Jenkins, Comp. J. 4 (1961), 47-53. See also the 
authors’ discussions of their own systems, which are articles of considerable 
historical importance: “An introduction to IPL-V” by A. Newell and F. M. Tonge, 
CACM 3 (1960), 205-211; “A FORTRAN-compiled List Processing Language” 
by H. Gelernter, J. R. Hansen, and C. L. Gerberich. JACM 7 (1960), 87 
101; “Recursive functions of symbolic expressions and their computation by 
machine, I” by John McCarthy, CACM 3 (1960), 184-195; “Symmetric List 
Processor” by J. Weizenbaum, CACM 6 (1963), 524-544. Weizenbaum’s article 
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included a complete description of all of the algorithms used in SLIP. Of all these 
early systems, only LISP had the necessary ingredients to survive the ensuing 
decades of further progress. McCarthy has described LISP’s early history in 
History of Programming Languages (Academic Press, 1981), 173-197. 

Several string manipulation systems also appeared during the 1960s; they 
were primarily concerned with operations on variable-length strings of alphabetic 
information — looking for occurrences of certain substrings and replacing them 
by others, etc. The most important of these from a historical perspective were 
COMIT [V. H. Yngve, CACM 6 (1963), 83-84] and SNOBOL [D. J. Farber, R. 
E. Griswold, and I. P. Polonsky, JACM 11 (1964), 21-30]. String manipulation 
systems were used widely, and they were composed primarily of algorithms like 
the ones we have seen in this chapter, but they played a comparatively small 
role in the history of the techniques of information structure representation; users 
of such systems were isolated from the details of the actual internal processes 
carried on by the computer. For a survey of early string manipulation techniques, 
see S. E. Madnick, CACM 10 (1967), 420-424. 

The IPL-V and FLPL systems for List-processing did not use either a garbage 
collection or a reference count technique for the problem of shared Lists; instead, 
each List was “owned” by one List and “borrowed” by all other Lists that referred 
to it, and a List was erased when its “owner” allowed it to disappear. Hence, the 
programmer was enjoined to make sure that no List was still borrowing any Lists 
that were being erased. The reference counter technique for Lists was introduced 
by G. E. Collins, CACM 3 (1960), 655-657, and explained further in CACM 9 
(1966), 578-588. Garbage collection was first described in McCarthy’s article of 
1960; see also Weizenbaum’s remarks in CACM 7 (1964), 38, and an article by 
Cohen and Trilling, BIT 7 (1967), 22-30. 

An increasing realization of the importance of link manipulations led natu- 
rally to their inclusion in algebraic programming languages designed after 1965. 
The new languages allowed programmers to choose suitable forms of data rep- 
resentation without resorting to assembly language or paying the overhead of 
completely general List structures. Some of the fundamental steps in this de- 
velopment were the work of N. Wirth and H. Weber [CACM 9 (1966), 13-23, 
25, 89-99]; H. W. Lawson [CACM 10 (1967), 358-367]; C. A. R. Hoare [Symbol 
Manipulation Languages and Techniques, ed. by D. G. Bobrow (Amsterdam: 
North-Holland, 1968), 262-284]; O.-J. Dahl and K. Nygaard [CACM 9 (1966), 
671-678]; A. van Wijngaarden, B. J. Mailloux, .J. E. L. Peck, and C. H. A. 
Koster [Numerische Math. 14 (1969), 79-218]; Dennis M. Ritchie [History of 
Programming Languages — II (ACM Press, 1996), 671-698]. 

Dynamic storage allocation algorithms were in use several years before they 
were ever described in print. A very readable discussion was prepared by W. T. 
Comfort in 1961 and published in CACM 7 (1964), 357-362. The boundary- 
tag method, introduced in Section 2.5, was designed by the author in 1962 for 
use in an operating system for the Burroughs B5000 computer. The buddy 
system was first used by H. Markowitz in connection with the SIMSCRIPT 
programming system in 1963, and it was independently discovered and published 


462 INFORMATION STRUCTURES 


2.6 


by K. Knowlton, CACM 8 (1965), 623-625; see also CACM 9 (1966), 616-625. 
For additional early discussions of dynamic storage allocation, see the articles 
by Iliffe and Jodeit, Comp. J. 5 (1962), 200-209; Bailey, Barnett, and Burleson, 
CACM 7 (1964), 339-346; A. T. Berztiss, CACM 8 (1965), 512-513; and D. T. 
Ross, CACM 10 (1967), 481-492. 

A general discussion of information structures and their relation to program- 
ming was prepared by Mary d’Imperio, “Data Structures and their Representa- 
tion in Storage,” Annual Review in Automatic Programming 5 (Oxford: Perga- 
mon Press, 1969). Her paper is a valuable guide to the history of the topic, since 
it includes a detailed analysis of the structures used in connection with twelve 
List processing and string manipulation systems. See also the proceedings of two 
symposia, CACM 3 (1960), 183-234 and CACM 9 (1966), 567-643, for further 
historical details. (Several of the individual papers from those proceedings have 
already been cited above.) 

An excellent annotated bibliography of early work on symbol manipulation 
and algebraic formula manipulation, having numerous connections with the ma- 
terial of this chapter, was compiled by Jean E. Sammet; see Computing Reviews 
7 (July-August 1966), B1-B31. 

In this chapter we have looked at particular types of information structures 
in great detail, and (lest we fail to see the forest for the trees) it is perhaps 
wise to take stock of what we have learned and to summarize briefly the general 
subject of information structures from a broader perspective. Starting with the 
basic idea of a node as an element of data, we have seen many examples that 
illustrate convenient ways to represent structural relationships either implicitly 
(based on the relative order in which nodes are stored in computer memory) or 
explicitly (by means of links in the nodes, which point to other nodes). The 
amount of structural information that ought to be represented within the tables 
of a computer program depends on the operations that are to be performed on 
the nodes. 

For pedagogic reasons, we have largely concentrated on the connections 
between information structures and their machine representations, instead of 
discussing those issues separately. However, to gain a deeper understanding it is 
helpful to consider the subject from a more abstract point of view, distilling off 
several layers of ideas that can be studied by themselves. Several noteworthy ap- 
proaches of this kind have been developed, and the following thought-provoking 
papers are especially recommended from the early literature: G. Mealy, “Another 
look at data,” Proc. AFIPS Fall Joint Computer Conf. 31 (1967), 525-534; 
J. Earley, “Toward an understanding of data structures,” CACM 14 (1971), 617- 
627; C. A. R. Hoare, “Notes on data structuring,” in Structured Programming 
by O.-J. Dahl, E. W. Dijkstra, and C. A. R. Hoare (Academic Press, 1972), 83- 
174; Robert W. Engles, “A tutorial on data-base organization,” Annual Review 
in Automatic Programming 7 (1972), 3-63. 

The discussion in this chapter does not cover the entire subject of informa- 
tion structures in full generality; at least three important aspects of the subject 
have not been treated here: 
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a) We often want to search through a table to find a node or set of nodes 
possessing a certain value, and the need for such an operation often has a 
profound effect on the structure of the table. This situation is explored in detail 
in Chapter 6. 

b) We have primarily been concerned with the internal representation of 
structure within a computer; but that is obviously only part of the story, since 
structure must also be represented in the external input and output data. In 
simple cases, external structure can be treated by essentially the same techniques 
that we have been considering; but the processes of converting between strings of 
characters and more complex structures are also very important. Those processes 
are analyzed in Chapters 9 and 10. 

c) We have primarily discussed representations of structures within a high- 
speed random-access memory. When slower memory devices such as disks or 
tapes are being used, we find that all of the structural problems are intensified; 
it becomes much more crucial to have efficient algorithms and efficient schemes 
for data representation. Nodes that link to each other in such cases ought to 
go into nearby areas of the memory. Usually the problems are highly dependent 
on the characteristics of individual machines, so it is difficult to discuss them in 
general. The simpler examples treated in this chapter should help to prepare the 
reader for solving the more difficult problems that arise in connection with less 
ideal memory devices; Chapters 5 and 6 discuss some of these problems in detail. 

What are the main implications of the subjects treated in this chapter? 
Perhaps the most important conclusion we can reach is that the ideas we have 
encountered are not limited to computer programming alone; they apply more 
generally to everyday life. A collection of nodes containing fields, some of which 
point to other nodes, appears to be a very good abstract model for structural 
relationships of all kinds. This model shows how we can build up complicated 
structures from simple ones, and we have seen that corresponding algorithms for 
manipulating the structure can be designed in a natural manner. 

Therefore it seems appropriate to develop much more theory about linked 
sets of nodes than we know at this time. Perhaps the most obvious way to 
start such a theory is to define a new kind of abstract machine or “automaton” 
that deals with linked structures. For example, such a device might be defined 
informally as follows: There are numbers k, l, r, and s, such that the automaton 
processes nodes containing k link fields and r information fields; it has l link 
registers and s information registers, which enable it to control the processes 
it is performing. The information fields and registers may contain any symbols 
from some given set of information symbols; each of the link fields and link 
registers either contains A or points to a node. The machine can (i) create 
new nodes (putting a link to the node into a register), (ii) compare information 
symbols or link values for equality, and (iii) transfer information symbols or link 
values between registers and nodes. Only nodes pointed to by link registers 
are immediately accessible. Suitable restrictions on the machine’s behavior will 
make it equivalent to several other species of automata. 
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A related model of computation was proposed by A. N. Kolmogorov as 
early as 1952. His machine essentially operates on graphs G, having a specially 
designated starting vertex v 0 . The action at each step depends only on the 
subgraph G' consisting of all vertices at distance < n from v 0 in G, replacing 
G' in G by another graph G" = /(G'), where G" includes « 0 and the vertices 
at distance exactly n from v 0 , and possibly other vertices (which are newly 
created); the remainder of graph G is left unaltered. Here n is a fixed number 
specified in advance for any particular algorithm, but it can be arbitrarily large. 
A symbol from a finite alphabet is attached to each vertex, and restrictions 
are made so that no two vertices with the same symbol can be adjacent to a 
common vertex. (See A. N. Kolmogorov, Uspekhi Mat. Nauk 8,4 (1953), 175- 
176; Kolmogorov and Uspensky, Uspekhi Mat. Nauk 13,4 (1958), 3-28; Amer. 
Math. Soc. Translations , series 2, 29 (1963), 217-245.) 

Linking automata can easily simulate graph machines, taking at most a 
bounded number of steps per graph step. Conversely, however, it is unlikely that 
graph machines can simulate arbitrary linking automata without unboundedly 
increasing the running time, unless the definition is changed from undirected to 
directed graphs, in view of the restriction to vertices of bounded degree. The 
linking model is, of course, quite close to the operations available to programmers 
on real machines, while the graph model is not. 

Some of the most interesting problems to solve for such devices would be 
to determine how fast they can solve certain problems, or how many nodes 
they need to solve certain problems (for example, to translate certain formal 
languages). At the time this chapter was first written, several interesting results 
of this kind had been obtained (notably by J. Hartmanis and R. E. Stearns) but 
only for special classes of Turing machines having multiple tapes and read/ write 
heads. The Turing machine model is comparatively unrealistic, so these results 
tended to have little to do with practical problems. 

We must admit that, as the number n of nodes created by a linking automa- 
ton approaches infinity, we don’t know how to build such a device physically, 
since we want the machine operations to take the same amount of time regardless 
of the size of n; if linking is represented by using addresses as in a computer 
memory, it is necessary to put a bound on the number of nodes, since the link 
fields have a fixed size. A multitape Turing machine is therefore a more realistic 
model when n approaches infinity. Yet it seems reasonable to believe that a 
linking automaton as described above leads to a more appropriate theory of 
the complexity of algorithms than Turing machines do, even when asymptotic 
formulas for large n are considered, because the theory is more likely to be 
relevant for practical values of n. Furthermore when n gets bigger than 10 30 
or so, not even a one-tape Turing machine is realistic: It could never be built. 
Relevance is more important than realism. 

Many years have passed since the author wrote most of the comments above, 
and everybody can be glad that substantial progress has indeed been made on 
the theory of linking automata (now called pointer machines ). But of course 
much still remains to be done. 
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General rules for programming have been discovered. 

Most of them have been used in the 
Kansas City freight yards for a long time. 

— DERRICK LEHMER (1949) 

You will, I am sure, agree with me . . . that if page 
534 finds us only in the second chapter, the length of 
the first one must have been really intolerable. 
— SHERLOCK HOLMES, in The Valley of Fear (1888) 


ANSWERS TO EXERCISES 


/ am not bound to please thee with my answers. 
— SHYLOCK, in The Merchant of Venice (Act IV, Scene 1, Line 65) 


NOTES ON THE EXERCISES 

1. An average problem for a mathematically inclined reader. 

4. See W. J. LeVeque, Topics in Number Theory 2 (Reading, Mass.: Addison- Wesley, 
1956), Chapter 3; P. Ribenboim, 13 Lectures on Fermat’s Last Theorem (New York: 
Springer- Verlag, 1979); A. Wiles, Annals of Mathematics 141 (1995), 443-551. 

SECTION 1.1 

1. t «— a, a t— 6, b <— c, c f— d, d <— t. 

2. After the first time, the values of the variables rn and n are the previous values of 
n and r, respectively; and n > r. 

3. Algorithm F ( Euclid’s algorithm). Given two positive integers m and n, find 
their greatest common divisor. 

FI. [Remainder m/n.} Divide m by n and let m be the remainder. 

F2. [Is it zero?] If m = 0, the algorithm terminates with answer n. 

F3. [Remainder n/m.} Divide n by m and let n be the remainder. 

F4. [Is it zero?] If n = 0, the algorithm terminates with answer m; otherwise go 
back to step FI. | 

4. By Algorithm E, n = 6099, 2166, 1767, 399, 171, 57. Answer: 57. 

5. Not finite nor definite nor effective, perhaps no output; in format, no letter is given 
before step numbers, no summary phrase appears, and there is no “|”. 

6. Trying Algorithm E with n = 5 and m = 1, 2, 3, 4, 5, we find that step El is 
executed 2, 3, 4, 3, 1 times, respectively. So the average is 2.6 = T s . 

7. In all but a finite number of cases, n > m. And when n > m, the first iteration of 
Algorithm E merely exchanges these numbers; so U m = T m + 1. 
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8. Let A = { a,b,c }, N = 5. The algorithm will terminate with the string a s cd ( m ' n ). 


3 


<t>j 

bj 

a j 


0 

ab 

(empty) 

1 

2 

Remove one a and one 6, or go to 2. 

1 

(empty) 

c 

0 

0 

Add c at extreme left, go back to 0. 

2 

a 

b 

2 

3 

Change all a’s to 6’s. 

3 

c 

a 

3 

4 

Change all c’s to a’s. 

4 

b 

b 

0 

5 

If 6’s remain, repeat. 


9. For example we can say C 2 represents Ci if there is a function g from h into I 2 , 
a function h from Q 2 into Q 1, and a function j from Q 2 into the positive integers, 
satisfying the following conditions: 

a) If x is in I\ then h.(g(x)) = x. 

b) If q is in Q 2 then = /i(/ 2 j(<7 " (<?)), where means that the function 

/ 2 is to be iterated j(q) times. 

c) If q is in Q 2 then h(q) is in fli if and only if q is in fl 2 . 

For example, let C 1 be as in ( 2 ) and let C 2 have I 2 = {(m, n)}, = {( m,n,d )}, 

Q 2 = h U SI 2 U {(m, n, a, b , 1)} U {(m, n, a, b , r, 2)} U {(ra, n, a, b, r, 3)} U {(m, n, a, b, 
r, 4)} U {(m, n, o, 6, 5)}. Let / 2 ((m, n)) = (m.,n,m,n, 1); f 2 ((m,n, d)) = ( to , n, d); 
h{(m, n,a, b, 1)) = (m, n, a, 6, a mod 6, 2); / 2 ((m, n, a, 6, r, 2)) = ( m,n,b ) if r = 0, 
otherwise (m,n,a,b,r, 3); / 2 ((m, n, a, b, r, 3)) = (m, n, b, b, r, 4); f 2 ((m,n,a,b,r, 4)) = 
{m,n,a,r, 5); f 2 ({m,n,a,b, 5)) = f 2 ((rn,n,a,b, 1)). 

Now let h((m,n)) = g((m,n )) = ( m,n ); h((m.,n,d)) = (d); h((m,n,a,b, 1)) = 
(a, b, 0,1); h((m,n,a,b,r, 2)) = (a, 6, r, 2); h((in,n,a,b,r/S )) = (a,b,r, 3); h((m, n, a, 
b,r, 4)) = h(/ 2 ((m, n, a, 6, r, 4))); h((m,n,a,b, 5)) = (a, 6,6, 1); j((m,n,a,b,r,3)) = 
j((m,n,a,b,r,4)) = 2, otherwise j(q) = 1. Then C 2 represents Ci. 

Notes: It is tempting to try to define things in a more simple way- for example, 
to let g map Qi into Q 2 and to insist only that when xo,xi,... is a computational 
sequence in Ci then g(xo),g(xi), ... is a subsequence of the computational sequence in 
C 2 that begins with g(a:o). But this is inadequate; in the example above, C 1 forgets 
the original value of m and n but C 2 does not. 

If C 2 represents C 1 by means of functions g , h, j , and if C3 represents C 2 by means 
of functions </, 6/, j 1 , then C3 represents C\ by means of functions g". h", j", where 

g"{x) = g'{g(x)), h!'{x) = h(h'(x)), and j"{q)= 3 ilk), 

0<k<j(h'(q)) 

if qo = q and q k+1 = f^ J < ' Qk ^(qk). Hence the relation defined above is transitive. We 
can say C 2 directly represents C\ if the function j is bounded; this relation is also 
transitive. The relation “C 2 represents Ci” generates an equivalence relation in which 
two computational methods apparently are equivalent if and only if they compute 
isomorphic functions of their inputs; the relation ll C 2 directly represents Ci” generates 
a more interesting equivalence relation that perhaps matches the intuitive idea of being 
“essentially the same algorithm.” 

For an alternative approach to simulation, see R. W. Floyd and R. Beigel, The 
Language of Machines (Computer Science Press, 1994), Section 3.3. 
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SECTION 1.2.1 

1. (a) Prove P( 0). (b) Prove that P( 0), .... P(n) implies P(n + 1), for all n > 0. 

2. The theorem has not been proved for n = 2; in the second part of the proof, take 
n = 1; we assume there that a 1 = 1. If this condition is true (so that a = 1), the 
theorem is indeed valid. 

3. The correct answer is 1 — 1/n. The mistake occurs in the proof for n = 1. when 
the formula on the left either may be assumed to be meaningless, or it may be assumed 
to be zero (since there are n — 1 terms). 

5. If n is prime, it is trivially a product of one or more primes. Otherwise n has 
factors, so n = km for some k and m with 1 < k,m < n. Since both k and m are less 
than n, by induction they can be written as products of primes; hence n is the product 
of the primes appearing in the representations of k and m. 

6. In the notation of Fig. 4, we prove A5 implies A 6. This is clear since A 5 implies 
(o' — qa)m + ( b ' — qb)n = ( am + b'n) — q(am + bn) = c — qd = r. 

7. n 2 — (n — l) 2 + • • • — ( — l) n l" = l + 2+ -- - + n = n(n + l)/2. 

8. (a) We must show that (n 2 - n + 1) + (n 2 - n + 3) -f h (n 2 + n - 1) equals 

n 3 : And indeed, the sum is n(n 2 - n) + (1 + 3 + • ■ ■ + (2n-l)) = n 3 - n 2 + n 2 , by 
Eq. ( 2 ). But an inductive proof was requested, so another approach should be taken! 
For n= 1, the result is obvious. Let n> 1; we have (n + 1 ) 2 - (n + 1) = n 2 - n + 2n, 
so the first terms for n + 1 are 2n larger; thus the sum for 77 . + 1 is the sum for n plus 

2n + ■ ■ ■ + 2 n+(n + l) 2 + (n + 1) — 1; 

V -v'" ^ 

n times 

this equals n 3 + 2 n 2 + r? 2 + 3n + 1 = (n + l) 3 . (b) We have shown that the first 
term for (71 + l) 3 is two greater than the last term for n 3 . Therefore by Eq. ( 2 ). 
1 +2 + • • • + 77 3 = sum of consecutive odd numbers starting with unity = (number of 
terms) 2 = (1 + 2 H + n) 2 . 

10. Obvious for n = 10. If n > 10, we have 2 n+1 = 2 ■ 2 n > (1 + l/n) 3 2" and by 
induction this is greater than (1 + l/n) 3 n 3 = (n + l) 3 . 

11. (-l) n (7z+ 1)/(4 (ti + l) 2 + 1). 

12. The only nontrivial part of the extension is the calculation of the integer q in E2. 
This can be done by repeated subtraction, reducing to the problem of determining 
whether u+vV ' 2 is positive, negative, or zero, and the latter problem is readily solved. 

It is easy to show that whenever u + v\/2 = u + t/\/ 2, we must have u = u' and 
v — v , since \/2 is irrational. Now it is clear that 1 and \/2 have no common divisor, 
if we define divisor in the sense that u + v^2 divides a(u + v\/2 ) if and only if a is an 
integer. The algorithm extended in this way computes the regular continued fraction 
of the ratio of its inputs; see Section 4.5.3. 

[Note: However, if we extend the concept of divisor so that u+vy/2 is said to divide 
(1 ( u + v \/2 ) if and only if a has the form u' -\-v' \/2 for integers u' and v', there is a way to 
extend Algorithm E so that it always will terminate: If in step E2 we have c = u + v\/2 
and d = u'+v'V 2, compute c/d = c{u' -v'y/2 )/{u' 2 -2v’ 2 ) = x+y ^ 2 where x and y are 
rational. Now let q = u" +u''v / 2, where u" and v" are the nearest integers to x and y, 
and let r = c — qd. If r = u'" + v'" y/2, it follows that |u"' 2 — 2v"' 2 \ < \u' 2 — 2v' 2 1, hence 
the computation will terminate. For further information, see “quadratic Euclidean 
domains” in number theory textbooks.] 
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13. Add “T < 3 (n — d) + k v to assertions AS, A4, A5, A6, where k takes the respective 
values 2, 3, 3, 1. Also add “d > 0” to A/. 

15. (a) Let A = S in (iii); every nonempty well-ordered set has a least element. 

(b) Let x -< y if |x| < |y| or if |x| = \y\ and x < 0 < y. 

(c) No, the subset of all positive reals fails to satisfy (iii). [Note: Using the so- 
called axiom of choice, a rather complicated argument can be given to show that every 
set can be well-ordered somehow; but nobody has yet been able to define an explicit 
relation that well-orders the real numbers.] 

(d) To prove (iii) for T n , use induction on n: Let A be a nonempty subset of T n 
and consider A\, the set of first components of A. Since Ai is a nonempty subset of S, 
and S is well-ordered, Ai contains a smallest element x. Now consider A x , the subset 
of A in which the first component equals x; A x may be considered a subset of T n _i 
if its first component is suppressed, so by induction A x contains a smallest element 
(x, X 2 , ■ . • , x„) that in fact is the smallest element of A. 

(e) No, although properties (i) and (ii) are valid. If 5 contains at least two distinct 
elements, a <b, the set (6), (a, b), (a, a, b ), (a, a, a, b), (a, a, a, a, b), . . . has no least ele- 
ment. On the other hand T can be well-ordered if we define (xi, . . . ,x m ) -< (yi, . . . ,y n ) 
whenever m < n, or m = n and (xi, . . . ,x„) -< (j/i, . . . ,y n ) in T n - 

(f) Let S be well-ordered by -<. If such an infinite sequence exists, the set A 
consisting of the members of the sequence fails to satisfy property (iii), for no element 
of the sequence can be smallest. Conversely if -< is a relation satisfying (i) and (ii) but 
not (iii), let A be a nonempty subset of S that has no smallest element. Since A is not 
empty, we can find xi in A; since xi is not the smallest element of A, there is X2 in A 
for which xi -< xi; since X2 is not the smallest element either, we can find xz < X 2 \ etc. 

(g) Let A be the set of all x for which P(x) is false. If A is not empty, it contains a 
smallest element x 0 . Hence P(y) is true for all y -< x 0 . But this implies P(x 0 ) is true, 
so xq is not in A (a contradiction). Therefore A must be empty: P(x) is always true. 


SECTION 1.2.2 

1. There is none; if r is a positive rational, rj 2 is smaller. 

2. Not if infinitely many nines appear in a row; in that case the decimal expansion 
of the number is 1 + .24000000 . . . , according to Eq. (2). 

3. —1/27, but the text hasn’t defined it. 

4. 4. 

6. The decimal expansion of a number is unique, so x = y if and only if m = n and 
di = e, for all i > 1. If x 7^ y, one may compare m vs. n, d\ vs. ei, ck vs. ea, etc.; when 
the first inequality occurs, the larger quantity belongs to the larger of {x,y}. 

7. One may use induction on x, first proving the laws for x positive, and then for x 
negative. Details are omitted here. 

8. By trying n = 0, 1, 2, . . . we find the value of n for which n m < u < (n + l) m . 
Assuming inductively that n,di, . . . ,dk-i have been determined, dk is the digit such 
that 


di 

n+ w + 


+ 


dk 

10^ 


< u < 


d 1 

n-\ + • 

10 


dk 1 

-I !L -| 

IQfc ^ IQ* 


9. ((bP/“) u /'’) qv = (((6P/«)“/“)'')« = ((&*/«)“)« = ((&P/«)«) U = b pu , hence (b p ^) u / v = 
b pu / q v. This proves the second law. We prove the first law using the second: b p ^ q b u ^ v = 
(b 1 / gv ) pv (b 1 / gv ) qu = (b 1 / qv ) pv+qu = b)P/t+ u / v 
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10. If log 10 2 = p/q, with p and q positive, then 2 q = 10 p , which is absurd since the 
right-hand side is divisible by 5 but the left-hand side isn’t. 

11. Infinitely many! No matter how many digits of x are given, we will not know 

whether 10* = 1.99999 . . . or 2.00000 . . . , if as’s digits agree with the digits of log 10 2. 
There is nothing mysterious or paradoxical in this; a similar situation occurs in addition, 
if we are adding .444444 ... to .55555 

12. They are the only values of d lt . . . , d s that satisfy Eq. (7). 

13. (a) First prove by induction that if y > 0, 1 + ny < (1 + y) n . Then set y = x/n. 
and take nth roots, (b) x = b - 1, n ~ 10 fc . 

14. Set x = log 6 c in the second equation of (5), then take logarithms of both sides. 

15. Prove it, by transposing “logf, y ” to the other side of the equation and using (11). 

16. lnx/ln 10, by (14). 

17. 5; 1; 1; 0; undefined. 

18. No, log 8 x = lga:/lg8 = | Ig x. 

19. Yes, since lgn < (log 10 n)/.301 < 14/. 301 < 47. 

20. They are reciprocals. 

21. (In In x — In In b)/ In b. 

22. From the tables in Appendix A, Igx « 1.442695 In a;; log 10 z « .4342945 Ins. The 
relative error is « (1.442695 - 1.4342945)/1.442695 « 0.582%. 

23. Take the figure of area In y, and divide its height by x while multiplying its length 
by x - This deformation preserves its area and makes it congruent to the piece left when 
In x is removed from In xy, since the height at point x + xt in the diagram for Inxy is 
l/(x + xt) = (1/(1 + t))/x. 

24. Substitute 2 everywhere 10 appears. 

25. Note that z = 2 P [2 P ' C x-J > 0, when p is the precision (the number of binary 
digits after the radix point). The quantity y + log 6 x stays approximately constant. 

27. Prove by induction on k that 

x\l - S ) 2fc+1 -r < 10 2 k ( n +bi/ 2 +-+b k / 2 k ) x , k < 3,2^ + , 

and take logarithms. 

28. The following solution uses the same auxiliary table as before. 

El. [Initialize.] If 1 - e is the largest possible value of x, set y (nearest 
approximation to ft 1 ”'), x 4- 1 - e - x, k <- 1. (The quantity yb~ x will 
remain approximately constant in the following steps.) 

E2. [Test for end.] If x = 0, stop. 

E3. [Compare.] If x < \og b {2 k /(2 k - 1)), increase k by 1 and repeat this step. 

E4. [Reduce values.] Set z <- * - log b (2 fc /(2 fc - 1)), y ^ y - (y shifted right k), 
and go to E2. | 

If y is set to b 1 E (1 + e 0 ) in step El, the subsequent computational error arises 
when x i x -f- log 6 (l - 2~ k ) + Sj and y i- y{ 1 - 2“ fc )(l + ey) during the jth execution 
of step E4, for certain small errors Sj and ej. When the algorithm terminates we have 
computed y = b x ll/ 1 + %)• Further analysis depends on b and the computer 
word size. Notice that both in this case and in exercise 26, it is possible to refine the 
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error estimates somewhat if the base is e, since for most values of k the table entry 

ln(2 k /(2 k — 1)) can be given with high accuracy: It equals 2~ k + \2~ 2k + 1 2~ 3k . 

Note: Similar algorithms can be given for trigonometric functions; see J. E. 
Meggitt, IBM J. Res. and Dev. 6 (1962), 210-226; 7 (1963), 237-245. See also T. C. 
Chen, IBM J. Res. and Dev. 16 (1972), 380-388; V. S. Linsky, Vychisl. Mat. 2 (1957), 
90-119; D. E. Knuth, METRFONT: The Program (Reading, Mass.: Addison- Wesley, 
1986), § 120— § 147. 

29. e; 3; 4. 

30. n. 


SECTION 1.2.3 


1. Ui + 0.2 + U3 . 

o 1 | 1 | 1 | 1 i 1 i 1 . 1 i 1 i 1 i 1 i 1 
** 1 i 3 I 5 i 7 T" g"i u ) 9 ' 3 ' l' 3 ' 9 ‘ 

3. The rule for p(j) is violated; in the first place, n 2 = 3 occurs for no n, and in the 
second place n 2 = 4 occurs for two n. [See Eq. (18).] 

4 . (an) + (a2i + 022) + (a 31 + 032 + 033) = (an + 021 + 031) + (022 + a 32) + (033). 

5. It is only necessary to use the rule a^ H ^ x, = J^R(i)( ax i) : 


R(i) 


S(j) 


J2 ai 

R(i) 


J2 b 3 

S(3 ) 


J2 ( a * b > 

H(0 V S(t) 


7. Use Eq. (3); the two limits are interchanged and the terms between a 0 and a c 
must be transferred from one limit to the other. 

8. Let a(i +1 )j = +1, and aj( <+1 ) = —1, for all i > 0, and all other zero; let 
R(i) — S(i) = “i > 0”. The left-hand side is —1, the right-hand side is +1. 

9. 10. No; the applications of rule (d) assume that n > 0. (The result is correct for 
ns -1 but the derivation isn’t.) 

11 . (n + l)a. 

12. |(1 — l/7 n+1 ). 

13. ra(n - m + 1) + |(n - m)(n - m + 1); or, |(n(n + 1) - m(m - 1)). 

14. \{n(n + 1) — m(m — l))(s(s + 1) — r(r — 1)), if m < n and r < s. 

15. 16. Key steps: 


Y 3 x3=x E & '=x Y U + ^x 3 

0<j<n 1 < j < n 0 < j < n — 1 


= x ^2 3 X 3 ~ nx n+1 + x 'Y x3 ■ 

0<j<n 0<j<n—l 

17. The number of elements in S. 

18. S'(j) = “1 < j < n”. R'(i,j) = “n is a multiple of i and i > j" . 

19. (a n — a m _i) [m < n], 

20. (b — 1) Yk=o( n ~ k)b k + n + 1 = this formula follows from (14) and the 

result of exercise 16. 

21 - E S (j ) a 3 + £s(j) a i = T,j a i [-R(i)] + a i [ s' (i ) ] = Y, a i([ R U)] + [ 5 0')]); now 
use the fact that [R(j)] + [S(j)] = [R{j) or S(j)] + [R(j) and S(j)\. In general, bracket 
notation gives us the ability to manipulate “on the line” instead of “below the line.” 
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22. For ( 5 ) and (7), just change £ to Y[. We also have n B(0 ha = (n« (0 MOW, d) 
and 

(n« 0 (n»-) = ( n ■><)( n <4 

X R(j) 7 V S(j) 7 'R(j) or S(j) ' 'R(j) and S(j) ' 

23. Q + x = x and 1 • x = x. This makes many operations and equations simpler, such 
as rule (d) and its analog in the previous exercise. 

25. The first step and last step are OK. The second step uses i for two different 
purposes at once. The third step should probably be £" =1 n. 

26. Key steps, after transforming the problem as in Example 2: 


n / n 

n n 

i=0 \j = 0 


CLidj 


nk +i ri' 


3= 0 





The answer is (n"=o a i) n+2 ■ 

28. ( n + l)/2 n. 

29. (a) E 0 <K,<i<„ di a i a k- (b) Let S r = J2 "=o a i- Solution: |S 3 + 5S1S2 + gS®. 
The general solution to this problem, as the number of indices gets larger, may be 
found in Section 1.2.9, Eq. (38). 


30. Write the left side as '£2 1 < :j j c < n a,jbkXjy k , and do a similar thing on the right. 
(This identity is the special case m = 2 of exercise 46.) 


31. Set a,j — Uj, bj — 1, x j — Vj , and yj = 1, to obtain the answer nW. 
33. This can be proved by induction on n, if we rewrite the formula as 


UjVj - 


1 ( x j( x 3 - 2n-l) __ Y' Xj(xj-X„) \ 

Xn-Xn^yjY ni< fc <„, W (*i-*t) k 11 l<k<n,k^j( X 3 ~ x k) )' 

Each of these sums now has the form of the original sum, except on n — 1 elements, 
and the values turn out nicely by induction when 0 < r < n — 1. When r = n, consider 
the identity 


0 _ nEi ( x j ~ x k) _y^ x j -i x 1 + 1- x n)x ] 1 + P(xj) 

j = 1 Hi <k<n, ~ x k) J =1 Tll<k<n,k^:j ( x 3 ~ x k) 

where P(xj) is a polynomial of degree n - 2; from the solution for r = 0, 1 , . . . , n - 1 
we obtain the desired answer. 

Notes. Dr. Matrix was anticipated in this discovery by L. Euler, who wrote to 
Christian Goldbach about it on 9 November 1762. See Euler’s Institutionum Calculi 
Integralis 2 (1769), §1169; and E. Waring, Phil. Trans. 69 (1779), 64-67. The following 
alternative method of proof, using complex variable theory, is less elementary but more 
elegant: By the residue theorem, the value of the given sum is 

f zr dz 

2ni J\z\=R (Z ~ Xi) . . . (z - X n ) 
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where R > |xi| , . . . , \x n \. The Laurent expansion of the integrand converges uniformly 
on \z\ = R\ it is 



r — n i / I i \ r — n — 1 , / 2 , . \ j — n — 2 , 

= 2 + (Xi + • • • + X n )Z + (x 1 + X\X 2 + ■ ■ • )Z +•••■ 

Integrating term by term, everything vanishes except the coefficient of z~ 1 . This 
method gives us the general formula for an arbitrary integer r > 0: 

T i« 

x i • • • x n • 

JlH hjn=r—n + 1 

3 1,-Jn >0 

[J. J. Sylvester, Quart. J. Math. 1 (1857), 141-152.] 

34. If the reader has tried earnestly to solve this problem, without getting the answer, 
perhaps its purpose has been achieved. The temptation to regard the numerators as 
polynomials in x rather than as polynomials in k is almost overwhelming. It would 
undoubtedly be easier to prove the considerably more general result 

r[l<r<n-l (?/* ~ Z r) 

22 — — = i, 

k= 1 ril<r<n, r^k(V k ~ V r ) 

which is an identity in 2 n — 1 variables! 

35. If R(j) never holds, the value should be — oo. The stated analog of rule (a) is based 
on the identity a + max(6,c) = max(a + 5, a + c). Similarly if all at, bj are nonnegative, 
we have 

SU Pi?(i) “i su Ps(j) b 3 = su Pi?(i) sup S y) a,bj. 

Rules (b), (c) do not change; for rule (d) we get the simpler form 

sup(sup R(j . ) aj, sup s(j) aj ) = sup R{j)oiSU) aj . 

36. Subtract column one from columns 2, . . . , n. Add rows 2, . . . , n to row one. The 
result is a triangular determinant. 

37. Subtract column one from columns 2, . . . ,n. Then subtract x\ times row k — 1 
from row k, for k = n, n — 1, . . . , 2 (in that order). Now factor x\ out of the first column 
and factor Xk — x\ out of columns k = 2, . . , , n, obtaining (*2 — *1 ) . . . ( x n — xi) times 
a Vandermonde determinant of order n — 1. The process continues by induction. 

Alternative proof, using “higher” mathematics: The determinant is a polynomial 
in the variables xi , . . . , x„ of total degree 1+2 + - • -+n. It vanishes if Xj = 0 or if x, = Xj 
( i < j), and the coefficient of x\x\. . .x„ is +1. These facts characterize its value. In 
general, if two rows of a matrix become equal for Xi = Xj, their difference is usually 
divisible by Xi — Xj, and this observation often speeds the evaluation of determinants. 

38. Subtract column one from columns 2, . . . ,n, and factor out 

(*1 + Zh)” 1 ■■■{x n + 2/i) _1 (yi - 2/2) • • • (l/i - Vn) 

from rows and columns. Now subtract row one from rows 2 ,...,n and factor out 
(xi —X 2 ) ■ ■ ■ ( xi ~Xn)(xi + i/2) _1 • • • (xi + Un)~ 1 ', we are left with the Cauchy determinant 
of order n — 1. 

39. Let I be the identity matrix (<5ij), and J the matrix of all ones. Since J 2 = nJ, 
we have (xi + y.J)((x + ny)I — yj) = x(x + ny)I. 
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40. [A. de Moivre, The Doctrine of Chances, 2nd edition (London: 1738), 197-199.] 
We have 

2^bitXj == Xj (xk — Xj) / Xi ( X/c — Xi) = Sij . 

t=l l<fc<n / l<fc<n 

k^i k^i 

41. This follows immediately from the relation of an inverse matrix to its cofactors. 
It may also be interesting to give a direct proof here: We have 


E 


i 


Xi + yt 


bt 7 — 


Y[k^t( x i + 2 Ik x )Ylk^i( x k + Vt) 

7^1 FI ~ x k)U k¥ :tiyt ~ Vk) 


when x = 0. This is a polynomial of degree at most n - 1 in x. If we set x = x 3 +y s , 
l<s<n, the terms are zero except when s = t, so the value of this polynomial is 

n<-‘* - / life- *0 = rife - xk- ~ x) / ij(xj - x k ). 

k 1 k ±3 / k^j 

These polynomials of degree at most n - 1 agree at n distinct points x, so they agree 
also for x = 0; hence 


E 


i 

Xi + yt 


btj 


life - Xk ) / rife - x *) = s n- 

k^i / k^j 


42. n/(x + ny). 

43 - 1 - FILO 1 - 1 /Xk). This is easily verified if any x t = 1, since the inverse of 
any matrix having a row or column all of ones must have elements whose sum is 1. 
If none of the x t equals one, sum the elements of row I as in exercise 44 and obtain 
llfc/ife ~ ~Xi). We can now sum this on i using exercise 33, with 

r = 0 (multiply numerator and denominator by (x< - 1)). 

44. We find 

n n j 

c i = e fe = rife + / n fe ~ **)> 

i= 1 fc= 1 / l<k<n 


after applying exercise 33. And 


E^E 


i x j + (yi + • • • + y n ) 


*?-' + ■ 


3=1 


3 = 1 


nr 


<k<n, fc^jfe 


Xk) 


- (x! + x 2 + b x n ) + (yi + 2/2 H b y n ). 

45. Let xt = i, y.j = j - 1. From exercise 44, the sum of the elements of the inverse is 

^ + • • • + n) + ((n — 1) + (n — 2) + • • • + 0) = n 2 . From exercise 38, the elements 
of the inverse are 


bin = 


(- 1) ,+J (7 + rr. - 1)! (j + n - 1)! 


(® + j ~ 1)(* — l)! 2 O' - l)! 2 (n — ?)! (n — j )! ' 

This quantity can be put into several forms involving binomial coefficients, for example 


y 


i+j - 1 
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From the latter formula we see that bij is not only an integer, it is divisible by i, j, n, 
i + j — 1, i + n — 1, j + n — 1, n — i + 1, and n — j + 1. Perhaps the prettiest formula 
for bij is 


(i + j 




The solution to this problem would be extremely difficult if we had not realized 
that a Hilbert matrix is a special case of a Cauchy matrix; the more general problem is 
much easier to solve than its special case! It is frequently wise to generalize a problem 
to its “inductive closure,” i.e., to the smallest generalization such that all subproblems 
that arise in an attempted proof by mathematical induction belong to the same class. In 
this case, we see that cofactors of a Cauchy matrix are Cauchy matrices, but cofactors 
of Hilbert matrices are not Hilbert matrices. [For further information, see J. Todd, 
J. Res. Nat. Bur. Stand. 65 (1961), 19-22; Cauchy, CEuvres (2) 12 , 173-182.] 

46. For any integers fci, k 2 , ... , k m , let e(ki , . . . , k m ) = sign(ni<«j< m (fcj -&*)), where 
signs = [x > 0] — [x < 0]. If (h , . . . , l m ) is equal to (fci, . . . , k m ) except for the fact that 
fcj and kj have been interchanged, we have e(Zi, . . . , l m ) = — e(fci, ...,k m ). Therefore 
we have the equation det (Bk 1 ...k m ) = e(fci, • • • ,k m ) det (Bj 1 ...j m ), if ji < • • • < jm are 
the numbers k \, . . . , k m rearranged into nondecreasing order. Now by definition of the 
determinant, 


det(AB) = Y e(Zi, • ■ 


( n \ 
Y^, a lkbkli I 

• • • 1 ^ (^mk^klm J 



Vk= i / 

Vfc= l / 

= y ] flifci • 

l<fcl ,...,fc m <n 

• • Q’mk rr 

. E 

1<1\ , . . • 1 1 m <m 

c(/i , . . . , /m) /j • • • ^k 

= y ] a,\k 1 ■ 

l<fcl <n 

• • Q , mk rT 

t det(J3fc 1 ...fc m 

) 

= Y 

• • 5 km , 

) O'lki • • • ® rnk-m 

. det 


l<fci , . . .,k m < n 


1<J1 < 


Finally, if two j’s are equal, det(ylj 1 ...j m ) = 0. [J. de l’Ecole Polytechnique 9 (1813), 
280-354; 10 (1815), 29-112. Binet and Cauchy presented their papers on the same day 
in 1812.] 

47. Let aij = (nU^+PtlKFlLj+it 11 i+rjk)). Subtract column k — 1 from column k 
and factor out Pk-j — <}k, for k = n, n — 1, ..., j + 1 (in that order), for j = 1, 
2, ..., n — 1 (in that order). This leaves YliK^jKniP' ~ Qi) ti mes det(fty) where 
bij = YYk=j+i{ x i + Ik)- Now subtract qk+j times column k + 1 from column k for 
k = 1, ..., n — j , and for j = 1, . . . , n — 1; this leaves det(c,j), where dj = x™ 3 
essentially defines a Vandermonde matrix. We can now proceed as in exercise 37, 
operating on rows instead of columns, obtaining 

det(a,j) = (xi - Xj)(pi - qj). 

l<.i<j <n 

When pj = qj = yj for 1 < j < n, the matrix in this exercise is a Cauchy matrix 
with row i multiplied by YYj=i( x i + Vj)- Therefore this result generalizes exercise 38 
by adding n — 1 independent parameters. [Manuscripta Math. 69 (1990), 177-178.] 
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SECTION 1.2.4 

1. 1, -2, -1, 0, 5. 

2. |zj. 

3 - By definition, [x\ is the greatest integer less than or equal to x; therefore |_xj is 
an integer, |_ x\ < x, and [xj + 1 > x. The latter properties, plus the fact that when m 
and n are integers we have m < n if and only if m < n - 1, lead to an easy proof of 
propositions (a) and (b). Similar arguments prove (c) and (d). Finally, (e) and (f) are 
just combinations of previous parts of this exercise. 

4. x — 1 < \_x\ < x; so —x + 1 > — [x\ > —x; hence the result. 

5 - l x + \\- The value of (-x rounded) will be the same as -( x rounded), except 
when x mod 1 = \ . In the latter case, the negative value is rounded towards zero and 
the positive value is rounded away from zero. 

6 •(f)) is true: Lv^J = n <(=> n 2 < x < (n + l) 2 n 2 < |xj < (n + l) 2 
LVL^J } = n - Similarly, (b) is true. But (c) fails when x is, say, 1.1. 

7. \x + y\ = |>J + x mod 1 + [yj +y mod lj = [x\ + [y\ + [x mod l + ymod lj. The 
inequality should be > for ceilings, and then equality holds if and only if either x or y is 
an integer or X mod 1 + y mod 1 > 1; that is, if and only if (- x ) mod 1 + (~y) mod 1 < 1. 

8. 1, 2, 5, -100. 

9. -1, 0, -2. 

10. 0.1, 0.01, -0.09. 

11. x = y. 

12. All. 

13. +1, -1. 

14. 8. 

15. Multiply both sides of Eq. (i) by z\ the result is also easily verified if y = 0. 

17. As an example, consider the multiplication portion of Law A: We have a = b + qm, 
and x = y + rm, for some integers q and r; so ax = by + (br + yq + qrm)m. 

18. We have a - 6 = kr for some integer fc, and also kr = 0 (modulo s). Hence by 
Law B, k = 0 (modulo s), so a — b = qsr for some integer q. 

20. Multiply both sides of the congruence by a. 

21. There is at least one such representation, by the previously proved exercise. If there 
are two representations, n = Pl . . .p k = qi . . . q m , we have q x ...q m = 0 (modulo Pl ); 
so if none of the q ' s equals pi we could cancel them all by Law B and obtain 1 = 0 
(modulo Pl ). The latter is impossible since Pl is not equal to 1. So some equals Pl , 
and n/pi = P2 ... Pk = qi . . . qj^ 1 qj +1 . . . q m . Either n is prime, when the result is 
clearly true, or by induction the two factorizations of n/ Pl are the same. 

22. Let m = ax, where a > 1 and x > 0. Then ax = 0 but x^0 (modulo m). 

24. Law A is always valid for addition and subtraction; Law C is always valid. 

26. If b is not a multiple of p, then b 2 - 1 is, so one of the factors must be. 

27. A number is relatively prime to p e if and only if it is not a multiple of p. So we 
count those that are not multiples of p and get tp(p e ) = p e — p e ~ 1 . 
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28. If a and b are relatively prime to m, so is ab mod m, since any prime dividing the 

latter and m must divide a or b also. Now simply let x \ , . . . , be the numbers 

relatively prime to m, and observe that ax\ mod m , . . . , ax mod m are the same 
numbers in some order, etc. 

29. We prove (b): If r _L s and if k 2 divides rs, then p 2 divides rs for some prime p, 
so p divides r (say) and cannot divide s; so p 2 divides r. We see that f(rs) = 0 if and 
only if f(r) = 0 or f(s) = 0. 

30 . Suppose ris. One idea is to prove that the (p(rs) numbers relatively prime to rs 
are precisely the ip(r)ip(s) distinct numbers (sx, + ryj) mod (rs) where xi , . . . ,x v ( r ) 
and 2 / 1 , ... , y v ( a ) are the corresponding values for r and s. 

Since tp is multiplicative, <^(10 6 ) = <p(2 6 )tp(5 6 ) = (2 6 - 2 5 )(5 6 - 5 s ) = 400000. 
And in general when n = p* 1 ...p® r , we have < p(n) = (pi 1 - p* 1 _1 ) . . . (p e r r - p e r r ~ l ) = 
n n p \„,pprime(l ~~ 1 /p)- (Another proof appears in exercise 1.3.3-27.) 

31 . Use the fact that the divisors of rs may be uniquely written in the form cd where 
c divides r and d divides s. Similarly, if f(n) > 0, one can show that the function 
ma x d \ n f(d) is multiplicative (see exercise 1.2.3-35). 

33 . Either n + morn-m + 1 is even, so one of the quantities inside the brackets is 
an integer; so equality holds in exercise 7, and we obtain (a) n; (b) n + 1. 

34 . b must be an integer > 2. (Set x = b.) The sufficiency is proved as in exercise 6. 
The same condition is necessary and sufficient for [log;,®] = [log 6 |"x]]. 

Note: R. J. McEliece has pointed out the following generalization: Let / be a 
continuous, strictly increasing function defined on an interval A, and assume that both 
[xj and |x] are in A whenever x is in A. Then the relation |_/(x)J = [/((xj)] holds 
for all x in A if and only if the relation |7(x)"| = f/(fasl)"| holds for all x in A, 
if and only if the following condition is satisfied for all x in A: “ f(x ) is an integer 
implies x is an integer.” The condition is obviously necessary, for if /(x) is an integer 
and it equals [/(IaJ)] or then x must equal [xj or |"x]. Conversely if, say, 

L/(kl)J < L/(z)J then by continuity there is some y with [xj < y < x for which f(y) 
is an integer; but y cannot be an integer. 


35 x + m _ 1 _ x + m _ 1 _ n - 1 [xj +m _ n - 1 < [xj + to I x + m 
n n n n n n ~ I n \ ~ n ’ 

apply exercise 3. Use of exercise 4 gives a similar result for the ceiling function. Both 
identities follow as a special case of McEliece’s theorem in exercise 34. 

36. Assume first that n = 2 1. Then 


hence 


E I =E 


k = 1 


n + 1 — k 




2t + 1 



by exercise 33. And if n = 2t + 1, we have t 2 + |n/2j = t 2 + t = n 2 / 4 - 1/4. For the 
second sum we get, similarly, \n(n + 2)/4]. 


478 


ANSWERS TO EXERCISES 


1 . 2.4 


37. 


E 

0 <fc<n 


mk + x 
n 


m(n — 1) 

2 


+ x. Let 


{?/} denote y mod 1; we must subtract 


5= E 

0 <fc<n 


mk + x 


This quantity S consists of d copies of the same sum, since if t = n/d we have 


mk + x \ / m(k + t) + x 


H 


Let u = m/d ; then 


E 


mk + x 


}- £ 

' n<r 


0 <k<t 


0 <k<t 

and since (in this sum may be rearranged to equal 
x mod d 


x ^ uk 
n t 


x mod d 1 
n t 


+ 


x mod d t ■ 




Finally, since ( x mod d)/n < 1 / 1. the braces in this sum may be removed and we have 


_ ^ f t(x mod d) t — 1 




')■ 


n 2 

An application of exercise 4 yields the similar identity 


E 

0 <k<n 


mk + x 


(m + l)(n — 1) d - 1 


+ d\x/d~\ . 


This formula would become symmetric in m and n if it were extended over the range 
0 < k < n. (The symmetry can be explained by drawing the graph of the summand as 
a function of k, then reflecting about the line y = x.) 

38. Both sides increase by |"y] when x increases by 1, so we can assume that 0 < x < 1. 
Then both sides are zero when x = 0, and both sides increase by 1 when x increases 
past the values 1 - k/y for y > k > 0. [Crelle 136 (1909), 42; the case y = n is due to 
C. Hermite, Act a Math. 5 (1884), 315.] 

39. Proof of part (f): Consider the more general identity rio<fc<n2 si n n ( x + k/n) = 
2 sin 7rna:, which can be demonstrated as follows: Since 2 sin# = ( e ie - e~ ie )/i = 
(1 — e~ 2ie )e t6 - ln / 2 , the identity is a consequence of the two formulas 

J~J (1 _ e ~^+ik/n)^ _ 1 _ e -2-rnx and ^(x-(\/2)+(k/n)) _ ^{nx-X/2) 

0<k<n 0 <k<n 


The latter is true since the function x — | is replicative; and the former is true because 
we may set z = 1 in the factorization of the polynomial z n — a" = (2 — a)(z — 00 a) . . . 
(z — w n ~ 1 a), where ui = e ~ 2,r, / T \ 

40. (Note by N. G. de Bruijn.) If / is replicative, f(nx + 1) - f(nx ) = f(x + 1) - f(x) 
for all n > 0. Hence if / is continuous, f(x + 1) - f(x) = c for all x, and g(x) = 
f(x) — c\x J is replicative and periodic. Now 

f e 2nlnx g(x) dx = - f e 2nlv g(y) dy\ 

Jo n Jo 
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expanding in Fourier series shows that g{x) = (x - \)a for 0 < x < 1. It follows 
that f(x) = (x — |)o. In general, this argument shows that any replicative locally 
Riemann-integrable function has the form (x- \)a+b max( |zj , 0)+cmin( [x\ , 0) almost 
everywhere. For further results see L. J. Mordell, J. London Math. Soc. 33 (1958), 371- 
375; M. F. Yoder, JEquationes Mathematical 13 (1975), 251-261. 

41. We want a n = k when |fc(fc — 1) < n < \k(k + 1). Since n is an integer, this is 
equivalent to 

fe ( fc ~ 1 ) + l <n< fc (fc + l) 1 

2 + 8 << 2 ' 8 ’ 

i.e., k — 1 < \/2 n < k + 1. Hence a n = [y/2n + | J, the nearest integer to y/2 n. Other 
correct answers are \y/2n- §] , |"(^8n+ 1 - l)/2], [(V8n - 7 + l)/2j, etc. 

42. (a) See exercise 1.2.7-10. (b) The given sum is n[log(,nJ — S, where 

5 = E k = E (b* -l) = (b ll ° s > ni+1 -b)/(b-l)-[\og b n\. 

l<fc<7l l<t<log b n 

fc+1 is a power of b 


43. [v^j(n-|(2LVHJ+5)(L^J-l))- 

44. The sum is n + 1 when n is negative. 


45. [mj/n\ = r if and only if 
sum is therefore 


rn 

m 


< j < 


(r + 1 )n 


, and we find that the given 


E /w 

0 < r < rn 



The stated result follows by rearranging the latter sum, grouping the terms with a 
particular value of \rn/m\. The second formula is immediate by the substitution 

f (x)=( X+ k 1 ). 

46 - Eo<i<an/(L”yV fl J) = T,o<r<am\ rn / m \(f( r - 1) - /W) + [an] /(Tam] - 1). 

47. (a) The numbers 2, 4, . . . , p — 1 are the even residues (modulo p); since 2kq = 
p[2kq/p\ + (2 kq) mod p, the number ( — i)L 2fc< r/?’J ((2 kq) inodp) will be an even residue 
or an even residue minus p, and each even residue clearly occurs just once. Hence 
(— 1) V p - 1)/2 2 • 4 . . . (p - 1) = 2 • 4 . . . (p - 1). (b) Let q = 2. If p = 4n + 1, a = n; if 
p = 4n + 3, a = n + 1. Hence (|) = (1, -1, -1,1) according as pmod8 = (1,3, 5, 7), 
respectively, (c) For k < p/4, we have 

[(p— 1 — 2k)q/p\ =q-\(2k + l)q/p] = q-l-[(2k + l)q/p\ = [{2k+l)q/p\ (modulo 2). 
Hence we may replace the last terms [(p - l)qr/pj, [(p - 3)q/p\,. . . by [q/p\, [3g/pJ, 

etc- (d) Eo<*< p/2 L*9/pJ + £o<r<,/ a rr P /*l = \p/i](\q/2\ - 1) = (p + l)(g - l)/4. 

Also Eo<r <,/2 \ r v/ O'] = Ylo<r<q /2 L TV t <?J + (<? — l)/2. The idea of this proof goes back 
to G. Eisenstein, Crelle 28 (1844), 246-248; Eisenstein also gave several other proofs 
of this and other reciprocity laws in the same volume. 

48. (a) This is clearly not always true when n < 0; when n > 0 it is easy to verify, 
(b) [(n + 2 — [n/25j)/3j = \(n- [n/25j)/3] = [(ra+ [-n/25])/3] = [[24n/25]/3] = 
[8n/25] = [(8n + 24)/25j. The penultimate equality is justified by exercise 35. 
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49 . Since /( 0) — /(/( 0)) — /(/( 0) + 0) = /( 0) + /( 0), we have f(n) = n for all 
integers n. If f(l) = k < 0, we have k = /(^/(f - k)) = - k)) = 

/(0) = 0. And if = 0 we have /(±) = f(±f( 1 + ^)) = f(^) = 0; 

furthermore 1 < m < n implies /(=*) = f(lf(2ZL)) = /(I) = o, for a = \n/m\, 
by induction on m. Thus /(±) < 0 implies f(x) = \x\ for all rational x. On the 
other hand, if /(±) > 0 the function g(x) = -/(-*) satisfies (i) and (ii) and has 
= 1 — /(§) < 0; hence f(x) = —g(—x) = — [— xj = [x] for all rational x. 
[P. Eisele and K. P. Hadeler, AMM 97 (1990), 475-477.] 

It does not follow, however, that /(x) = |xj or [x] for all real values of x. If, for 
example, h(x) is any function with h(l) = 1 and h(x + y) - h(x) + h(y) for all real x 
and y, then the function /(x) = [h(x)\ satisfies (i) and (ii); but h(x) may be unbounded 
and highly erratic when 0 < x < 1 [G. Hamel, Math. Annalen 60 (1905), 459-462]. 


SECTION 1.2.5 


1. 52!. For the curious, this number is 806 58175 17094 38785 71660 63685 64037 
66975 28950 54408 83277 82400 00000 00000. (!) 

2. p n k = p n (k-i)(n — fc + 1). After the first n — 1 objects have been placed, there is 
only one possibility for the last object. 

3 . 53124,35124,31524,31254,31245; 42351,41352,41253,31254,31245. 

4 . There are 2568 digits. The leading digit is 4 (since log 10 4 = 2 log 10 2 « .602). The 

least significant digit is zero, and in fact by Eq. (8) the low order 249 digits are all zero. 
The exact value of 1000! was calculated by H. S. Uhler using a desk calculator and much 
patience over a period of several years, and appears in Script a Mathematica 21 (1955), 
266-267. It begins with 402 38726 00770 .... (The last step in the calculation, to 
multiply the two numbers 750! and was performed on UNIVAC I by John W. 

Wrench, Jr., “in the extraordinary time of 21/2 minutes.” Nowadays, of course, a 
desktop machine easily produces 1000! in a fraction of a second, and we can confirm 
that Uhler’s value was 100% correct.) 

5 . (39902) (97/96) « 416 + 39902 = 40318. 

6. 2 18 • 3 8 • 5 4 • 7 2 • 11 • 13 • 17 • 19. 

8. It is limm-Kx, m"m!/((n + m)\/n\) = n\ lim m ^ 00 m n /((m + 1) . . . (m + n)) = n!, 
since m/(m + k) — » 1. 

9. yfn and —2^/tt. (Exercise 10 used.) 

10 . Yes, except when x is zero or a negative integer. For we have 


r(x + 1) = x lim 


m—yoo x(x + 1) . . . (x + m) \X + m + 


( m Y 

Vx + m + 1 / 


11,12. p — (akp k 1 + • ■ • + ai) + (akp k 2 + • ■ • + 012 ) + • • • + a*, 

= ak{p k 1 H \-p+ 1) H bai = ( ak(p k — 1)4 hao(p° — l))/(p— 1) 

= (n - ak ai - ao)/(p— 1). 

13 . For each n, 1 < n < p, determine n! as in exercise 1.2.4-19. There is exactly 
one such n', by Law 1.2.4B; and (n')' = n. Therefore we can pair off the numbers in 
groups of two, provided that n! / n. If n = n, we have n 1 = 1 (modulo p); hence, as 
in exercise 1.2.4-26, n = 1 or n = p - 1. So (p - 1)! = 1 ■ 1 . . . 1 . (-1), since 1 and p - 1 
are the only unpaired elements. 
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14. Among the numbers {1,2, ...,n} that are not multiples of p, there are [n/p\ 
complete sets of p — 1 consecutive elements, each with a product congruent to — 1 
(modulo p) by Wilson’s theorem. There are also oo left over, which are congruent 
to a 0 ! (modulo p); so the contribution from the factors that are not multiples of p is 
(_l)L"/pJ ao ! The contribution from the factors that are multiples of p is the same as 
the contribution in [n/pj!; this argument can therefore be repeated to get the desired 
formula. 


15. (n!) 3 . There are n\ terms. Each term has one entry from each row and each 
column, so it has the value (n!) 2 . 


16. The terms do not approach zero, since the coefficients approach 1/e. 

17. Express the gamma functions as limits by Eq. (15). 

18. TT — — = r ^) r (f) _ 2 r(l) 2 

!>i n-\n+\ r(l)T(l) ^ ' 

[Wallis’s own heuristic “proof” can be found in D. J. Struik’s Source Book in Mathe- 
matics (Harvard University Press, 1969), 244-253.] 

19. Change of variable t = mt, integration by parts, and induction. 


20. [For completeness, we prove the stated inequality. Start with the easily verified 
inequality 1 +x < e 1 ; set x = ±t/n and raise to the nth power to get (1 ±t/n) n < e ±( . 
Hence e“* > (\-t/n) n = e -i (l-t/n)V > e _t (l-f/n) n (l + f/n) n = e _t (l -t 2 /n 2 ) n > 
e -t (l — t 2 /n) by exercise 1.2. 1-9.] 

Now the given integral minus r m (ic) is 


f 


e~ t t x ~ 1 dt + 


)>"* 


As m — » 00, the first of these integrals approaches zero, since t x 1 < eJ^ 2 for large f; 
and the second is less than 

1 pm 1 poo 

— / t x+1 e~ t dt < — / t x+1 e~ t dt — > 0. 
m Jo m Jo 

21. If c(n,j, /ci, fc 2 , ■ • • ) denotes the appropriate coefficient, we find 

c(n+l, j, fci, . . . ) = c(n,j- 1, ki — l,k 2 ,.. . ) + (fci + l)c(n, j, ki + 1, k 2 - 1, k 3 , . . . ) 

+ {k 2 + 1) c(n, j, ki, k 2 + 1, k 3 - 1, k 4r . . . ) 4 , 

by differentiation. The equations k\ + k 2 + • • • = j and k\ + 2k 2 + ■ ■ • = n are preserved 
in this induction relationship. We can easily factor n!/(fc x ! (l!) fcl fc 2 ! (2!)*’ 2 ...) out of 
each term appearing on the right-hand side of the equation for c(n + l,j,ki, . . .), and 
we are left with ki + 2 k 2 + 3&3 + • • • = n + 1. (In the proof it is convenient to assume 
that there are infinitely many fc’s, although clearly k n+i = k n+2 = ■ ■ ■ = 0.) 

The solution just given makes use of standard techniques, but it doesn’t give 
a satisfactory explanation of why the formula has this form, nor how it could have 
been discovered in the first place. Let us examine this question using a combinatorial 
argument suggested by H. S. Wall [Bull. Amer. Math. Soc. 44 (1938), 395-398]. Write 
for convenience Wj = D J u w, u k = D k x u. Then D x ( Wj ) = w j+ iu x and D x {u k ) = u k +i- 
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By these two rules and the rule for derivative of a product we find 

D].w = W 1 U 1 

Dl w = ( W 2 U 1 U 1 + W 1 U 2 ) 

D 3 x w = {{w 3 U\U\u\ + W 2 U 2 U 1 + W 2 U 1 U 2 ) + {W 2 U 1 U 2 + W 1 U 3 )), etc. 
Analogously we may set up a corresponding tableau of set partitions thus: 

^ = { 1 } 

D 2 = ({2}{1} + {2,1}) 

= (({3}{2}{1} + {3, 2}{1} + {2}{3, 1}) + ({3}{2, 1} + {3, 2, 1})), etc. 

Formally, if aio 2 . . . aj is a partition of the set {1, 2, . . . , n - 1}, define 

Vaia 2 . . . a.j = {n}aia 2 . . . ctj + (ai U {n})a 2 . . . a,j 

+ ai(a 2 U {n}) . . . aj + ■ • • + aio 2 . . . ( a,j U {n}). 

This rule is an exact parallel of the rule 

D x (WjU ri U r 2 ...U rj ) = Wj + iUlU ri Ur 2 . . . U Tj + WjU ri + lU r2 . . . U Tj 

+ WjU ri U r2 - )-i . . ,U rj + 1- WjU ri U r2 . . .U rj + 1 , 

if we let the term WjU ri u r2 ■ ■ ■ u rj correspond to a partition aia 2 . . . a 3 with r t elements 
in at, 1 < t < j. So there is a natural mapping from V n onto D x w, and furthermore 
it is easy to see that V n includes each partition of the set {1,2,..., n} exactly once. 
(See exercise 1.2.6-64.) 

From these observations we find that if we collect like terms in D x w, we obtain a 

sum of terms c(ki ,k 2 ,... )wju\ 1 u 2 2 . . . , where j = k 1 + k 2 -\ and n = k\ + 2 k 2 4 , 

and where c(fci, fc 2 , . . . ) is the number of partitions of {1,2, ... , n} into j subsets such 
that there are kt subsets having t elements. 

It remains to count these partitions. Consider an array of k t boxes of capacity t: 

k 3 , — , 

k 2 ' * -M 

fci ftpt 

□□...□DC 



The number of ways to put n different elements into these boxes is the multinomial 
coefficient 


/ n \ _ n! 

V 1,1,..., 1,2, 2,..., 2, 3, 3,..., 3, 4 ,.. .) ~ l!*i2!*23! fc 3 . . 

To get c(ki,k 2 ,k 3 , . . . ) we should divide this by fci! k 2 \ k 3 \ . . . , since the boxes in each 
group of k t are indistinguishable from each other; they may be permuted in k t \ ways 
without affecting the set partition. 

Arbogast’s original proof [Du Calcul des Derivations (Strasbourg: 1800), §52] was 
based on the fact that D x u/k! is the coefficient of z k in u(x + 2 ) and D 3 u w/j\ is the 
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coefficient of y 3 in w(u + y), hence the coefficient of z n in w(u(x + z)) is 
v j\ ( Dlu \ kl ( Dlu\ k2 

n\ f-; j\ fci!fc 2 !...* n ! V 1! ) V 2! ) 

3 ~° k 1 +k 2 + -+k n =j 

k i -f 2k2-\ 1 -nk n =n 

k i ,fc2,...,fc n >0 

His formula was forgotten for many years, then rediscovered independently by F. Faa 
di Bruno [Quarterly J. Math. 1 (1857), 359-360], who observed that it can also be 
expressed as a determinant 



/("o>- 


(VK •• 

. 

(»=*)«» \ 


-1 

(VK 

(V)u 2 .. 

• CzlW- 2 


-D" = det 

0 

-1 

( n oV •• 

■ (”=2)«»-s 

CZlhn-2 


V o 0 0 ... -1 Or / 


where Uj = ( D 3 x u ) D u \ both sides of this equation are differential operators to be applied 
to w. For a generalization of Arbogast’s formula to functions of several variables, and 
a list of references to other related work, see the paper by I. J. Good, Annals of 
Mathematical Statistics 32 (1961), 540-541. 

22. The hypothesis that lim n - +00 (n+a;)!/ (n! n x ) = 1 is valid for integers x; for example, 
if x is positive, the quantity is (l + l/n)(l+2 /n) . . . (1+x/n), which certainly approaches 
unity. If we also assume that x\ = x(x — 1)!, the hypothesis leads us to conclude 
immediately that 



1 = lim = xl lim i£+ 1 )---( a + ^) ; 

n— >00 n! n x n — >oo n! n x 

which is equivalent to the definition given in the text. 

23 . z (— z)\ r (z) = limm-Hx, niT=r(l _ 2 / n ) _1 (1 + z / n )^ 1 by (13) an d (15). 

24 . n n /n\ = fl lZ\(k + l) k /k k < ri\/n ” +1 = U.V1 k k+1 / (k + l ) k+1 < 

25 . x^ 1 ^ = x—(x — m)-; x m+n = x m (x + m) n . These laws hold also when m and n 
are nonintegers, by (21). 


SECTION 1.2.6 

1. n, since each combination leaves out one item. 

2 . 1. There’s exactly one way to choose nothing from the empty set. 

3. (^3). The actual number is 635013559600. 

4. 2 4 • 5 2 • 7 2 • 17 • 23 • 41 ■ 43 ■ 47. 

5. (10 + l) 4 = 10000 + 4(1000) + 6(100) + 4(10) + 1. 


6. r = —3: 

1 

-3 

6 

-10 

15 

-21 

28 

-36 

r = —2: 

1 

-2 

3 

-4 

5 

-6 

7 

-8 

r = —1: 

1 

-1 

1 

-1 

1 

-1 

1 

-1 


7. L n /2J; or, alternatively, [n/2]. It is clear from (3) that for smaller values the 
binomial coefficient is strictly increasing, and afterwards it decreases to zero. 
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8. The nonzero entries in each row are the same from left to right as from right to left. 

9. One if n is positive or zero; zero if n is negative. 

10 - ( a ), ( b ) an d (f) follow immediately from (e); (c) and (d) follow from (a), (b), and 
Eq. (9). Thus it suffices to prove (e). Consider (£) as a fraction, given by Eq. (3) with 
factors in numerator and denominator. The first k mod p factors have no p's in the 
denominator, and in the numerator and denominator these terms are clearly congruent 
to the corresponding terms of 

/ n mod p \ 

V k mod p ) ’ 

which differ by multiples of p. (When dealing with non-multiples of p we may work 
modulo p in both numerator and denominator, since if a = c and b = d and 0/6, c/d 
are integers, then a/b = c/d.) There remain k-k modp factors, which fall into [k/p\ 
groups of p consecutive values each. Each group contains exactly one multiple of p: 
the other p - 1 factors in a group are congruent (modulo p) to (p - 1)! so they cancel 
in numerator and denominator. It remains to investigate the [k/p] multiples of p in 
numerator and denominator; we divide each of them by p and are left with the binomial 
coefficient 

/ |_(ra ~ k mod p) /p\ \ 

V l k /p\ J ' 

If k mod p < n mod p, this equals 

( V n /p\ \ 

V i k /p\ ) 

as desired; and if k mod p > n mod p, the other factor (" p ) is zero, so the formula 
holds in general. [American J. Math. 1 (1878), 229-230; see also L. E. Dickson, Quart. 
J. Math. 33 (1902), 383-384; N. J. Fine, AMM 54 (1947), 589-592.] 

11. If a = a r p + • • • + ao , & = br-p r + • ■ • + 60 , and a + 6 = c r p r -)- • • • -(- co , the value 
of n (according to exercise 1.2.5-12 and Eq. (5)) is 

(do + ■ • • + a r + bo + • • • + b r — co — • ■ ■ — c r )/{p — 1). 

A carry decreases Cj by p and increases c J+ 1 by 1, giving a net change of +1 in this 
formula. [Similar results hold for g-nomial and Fibonomial coefficients; see Knuth and 
Wilf, Crelle 396 (1989), 212-219.] 

12. By either of the two previous exercises, n must be one less than a power of 2. More 
generally, (”) is never divisible by the prime p, 0 < k < n, if and only if n = ap m - 1, 
1 < a < p, m > 0. 

“•KTV-m+KTMT) 

_ n 5 n 4 ra 3 n _ n(n + l)(n + |)(3n 2 + 3n - 1) 
~5 + 2 + T - 30 15 ’ 

15. Induction and (9). 

17. We may assume that r and s are positive integers. Also 

= r- = £(;>‘£0" 

n km 

=E(i>*E(„: t >"-‘-E(Ea)(„: t ))«" 

for all x, so the coefficients of x n must be identical. 
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21. The left-hand side is a polynomial of degree < n; the right-hand side is a polyno- 
mial of degree m + n + 1. The polynomials agree at n + 1 points, but that isn’t enough 
to prove them equal. [In fact, the correct formula in general is 


Y^/r-fc\/s + fc\ _ / r + s+1 \ 
f—' V m ) V n ) Vm + n + 1/ 


f(r 


)( 


m + n + 1 — k , 


when m, n, and r are nonnegative integers.] 
22. Assume n > 0. The fcth term is r times 


n (r-ffc-j) n (' n-l-r + tk-j ) 

0 <j<k 0 <j<n-k 

= (l) FI (-r + tk+j) (-r + tk+j) 

0 <j<k k<.j<n 

and the two products give a polynomial of degree n — 1 in k, so the sum over k is zero 
by Eq. (34). 

24. The proof is by induction on n. If n < 0 the identity is obvious. If n > 0, we 
prove it holds for (r,n — r + nt + m, t , n), by induction on the integer m > 0, using the 
previous two exercises and the validity for n - 1. This establishes the identity (r, s, t, n) 
for infinitely many s, and it holds for all s since both sides are polynomials in s. 

25. Using the ratio test and straightforward estimates for large values of k we can 
prove convergence. (Alternatively using complex variable theory we know that the 
function is analytic in the neighborhood of x = 1.) We have 


k,J 3 k 


= E^E('7 , )( r 7-J J K = EM)^(MXi + 


V. 


Now let x = l/(l + w), z = — w/(l + w) 1+t . This proof is due to H. W. Gould [AMM 63 
(1956), 84-91]. See also the more general formulas in exercises 2.3.4.4-33 and 4.7-22. 

26. We could start with identity (35) in the form 


i>or/) 


and proceed as in exercise 25. Another way is to differentiate the formula of that 
exercise with respect to z\ we get 


^ kA k (r, t)z k 

k 


d(x r ) _ (x t+1 — x t )rx r 

dz (t + l)x t+1 — tx 4 ’ 


hence we can obtain the value of 

^(f - Ak(r,t)z k . 

k 


27. For Eq. (26), multiply the series for x r+1 /((t + l)a; — t) by the series for x s , and 
get a series for x r+s+1 /((t + l)x — t ) in which coefficients of z may be equated to the 
coefficients arising from the series for a:^ r+s ^ +1 /((f + l)a: — t). 
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28. Denoting the left-hand side by f(r,s,t,n ), we find 

( n ) +tf(r + t - 1, s - t, t, n - 1) = f(r,s,t,n) 
by considering the identity 

29 - (_1) ' c (fc)/ n! = ( “ 1)fc/(fc!(n “ fc)!) = (_1)n / n (*-./)• 

30. Apply (7), (6), and (19) to get 

w 


0<j <n 


k> 0 


/ — m — 2 k — 1\ 

( 2 k + 1 \ 

V n — m — k J 

V k ) 


Now we can apply Eq. (26) with ( r,s,t,n ) = (1, m - 2n - 1, -2, n - m), obtaining 

^ j \n — m f ^ \ ( Tl 1 \ 

V n — mj \n — m) 

This result is the same as our previous formula, when n is positive, but when n = 0 the 
answer we have obtained is correct while (^_\) is not. Our derivation has a further 
bonus, since the answer ( "“^) is valid for n > 0 and all integers m. 

31. [This sum was first obtained in closed form by J. F. Pfaff, Nov a Acta Acad. Scient 
Petr. 11 (1797), 38-57.] We have 


k j 


j k 


/ m - r + s\ 

/ n + r — s\ 

( r ) 

I'm — r + s — j\ 

y j ) 

V n — k ) 

\m + n — j ) 

V k-j ) 


U 


/ m — r + s\ 

( V ) 

( m + n-j\ 

V j ) 

\m + n — j J 

V n - j J 


Changing J ) to ( m+ ^ j ) and applying (20) again, we get 

e(’"7 + ')(;,)(;:7)=(;)(;)^ 

32. Replace x by —x in (44). 

33, 34. [Mem. Acad. Roy. Sci. (Paris, 1772), part 1, 492; C. Kramp, Elemens d’Arith- 
metique Universelle (Cologne: 1808), 359; Giornale di Mat. Battaglini 33 (1895), 179- 
182.] Since x n = n! (‘ r+ " x ), the equation may be transformed into 

/'® + 2/ + n- l\_, i p/x+(l-z)A;\ f y - \ + nz + (n - k)(l - z)\ x 
^ n ' * V k J\ n-k ) x + (1 — z)k’ 

which is a case of (26). Similarly, (x + y )2 = (^)a:(x - kz - l)^(jy + kz ) a =*, an 

equivalent formula of Rothe [Formula; de Serierum Reversione (Leipzig: 1793), 18], 

35. For example, we prove the first formula: 


£(-D 

k 


n+1 — k 


n 

L/c - lj 


x k = — nx~ + xx- = x 7 ^! . 
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36 . By (13), assuming that n is a nonnegative integer, we get 2 n and 5 n0 , respectively. 

37 . When n > 0, 2"~ 1 . (The odd and even terms cancel, so each equals half the total 
sum.) 

38 . Let lj = e ' 27ri/rn . Then 


E (i+«T^' fc = E E (")^ ( 


t 0<j<m 


uj rj = m[r = 0 (modulo m)] 


(it is the sum of a geometric progression), so the right-hand sum is m ^ ( ”)' 


The original sum on the left is 


<uj - j/2 +cj j/ 2 ) n oo j( - n/2 - k) 


- E (»«■£)’ 


J( n/2-k) 


Since the quantity is known to be real, we may take the real part and obtain the stated 
formula. [See Crelle 11 (1834), 353-355.] 

The cases m = 3 and m = 5 have special properties discussed in CMath , exercises 
5.75 and 6.57. 

39 . n!; < 5 n0 — Sni- (The row sums in the second triangle are not so simple; we will find 
(exercise 64) that J2k {£} ‘ s the number of ways to partition a set of n elements into 
disjoint sets, which is the number of equivalence relations on (1,2 ,..., n}.) 

40 . Proof of (c): By parts, 

B(* + 1, y) = 1 + - f 1 1 * _1 (1 - t) v dt. 

0 y Jo 


Now use (b). 

41 . m x B(a;, m + 1) — > r(s) as m — > 00, regardless of whether m runs through integer 
values or not (by monotonicity). Hence, (m + y) x B(x , m + y + 1) — ► T(x), and 
(m/(m + y)) x 1. 

42 . l/((r + l)B(/c + 1 , r — k + 1)), if this is defined according to exercise 41(b). In 
general when z and w are arbitrary complex numbers we define 

( Z ^ — 1;™ U™ ^ ‘ ...u /■( T't r I 


lim lim ' — -r, 

U-M1 io\ (Q — UJ)\ 


where = T(^ + 1); 


the value is infinite when z is a negative integer and w is not an integer. 

With this definition, the symmetry condition (6) holds for all complex n and k, 
except when n is a negative integer and k is an integer; Eqs. (7), (9), and (20) are 
never false, although they may occasionally take indeterminate forms such as 0 • 00 or 
00 T 00. Equation (17) becomes 

/ 2 \ _ sin7r(w — 2 — 1) f w — z — 1\ 


o- 


We can even extend the binomial theorem (13) and Vandermonde’s convolution (21), 
obtaining Y, k L+ fc )z“ +fe = (1 + z) r and J2 k L+Jip-k) = (a+ia); these formulas hold 
for all complex r, s, z, a, and f3 whenever the series converge, provided that complex 
powers are suitably defined. [See L. Ramshaw, Inf. Proc. Letters 6 (1977), 223-226.] 


1.2.6 


488 ANSWERS TO EXERCISES 


43. f Q dt/(t 1 ^~( 1 — t) 1 / 2 ) — 2 f* du/( 1 — u 2 ) 1 / 2 = 2 arcsin u|J = n. 

1 


45. For large r, 


r 1 (1 — k/r) k 


kT(k) Y r — k e k (1 — k/r) r F(k + 1) * 

46 ' vSFv) (‘ + !)'(’ + ;)’• “ d O * 

47. Each quantity is <5 fc0 when k < 0, and is multiplied by (r - k)(r - A - k)/(k + l) 2 

when k is replaced by k + 1. When r = - \ this implies = (-1/4)* ( 2k ) . 

48. This can be proved by induction, using the fact that 


» = LC)(-.)* = E(;)^ + E(I) 


n \ (— 1)* 


k + x 


when n > 0. Alternatively, we have 

Birr, n + 1 ) = £ t*~\ 1 - t) n dt = £ (™) (-l) fc jf ' t x+k ~ l dt. 

(In fact, the stated sum equals B(a:, n+1) for noninteger n also, when the series 
converges.) 

49 - (1) = e(I) ( m - r 2 J ( “ i)m+,; ’ integer m - (See exercise i7 -) 

k 

50. The fcth summand is (”) (-l) n ~ k (x - kz) n ~ 1 x. Apply Eq. (34). 

51. The right-hand side is 


J2( n -k) x ^ x ~ kz ^ j k )(* + yY(-x + kz) n 


-k-j 


= E ” <*+»>'£ 


n -3 
n — j — k 


x{x-kzt~ 1 (-x + kz) n - k - i 


Y1 ( U j ) + yY° n 3 = (* + 2 /)" 

j<n 


The same device may be used to prove Torelli’s sum (exercise 34). 

Another neat proof of Abel’s formula comes from the fact that it is readily trans- 
formed into the more symmetric identity derived in exercise 2.3.4.4-29: 

^ 2 (JI) X ( X + kz ) k ~ 1 y(y+ {n ~ k)z ) n ~ k ~ 1 = (x + y)(x + y + nz)™^ 1 . 


Abel’s theorem has been generalized even further by A. Hurwitz [Acta Mathema- 
tics 26 (1902), 199-203] as follows: 

^2 x (x + eiZ! + ■ ■ ■ + e n Zn) ei+ +6n 1 (y — eizi — ■ ■ ■ - € n z n ) n ~ ei e ” = (x + y) n 

where the sum is over all 2" choices of ei, . . . , e n = 0 or 1 independently. This is an 
identity in x, y, zi, . . . , z„, and Abel’s formula is the special case zi = z 2 = ■ ■ ■ = z n . 
Hurwitz’s formula follows from the result in exercise 2.3.4.4-30. 
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52. E fc >o(fc + 1) 2 = 7 t 2 /6. [M. L. J. Hautus observes that the sum is absolutely 
convergent for all complex x, y, z, n whenever 2 / 0, since the terms for large k 
are always of order 1/A: 2 . This convergence is uniform in bounded regions, so we may 
differentiate the series term by term. If f(x, y, n ) is the value of the sum when 2 = 1, we 
find (d/dy)f(x,y,n) = nf(x, y,n- 1) and (d/dx)f{x, y, n) = nf(x- l,y + l,n- 1). 
These formulas are consistent with f(x, y, n) = ( x + y) n ; but actually the latter equality 
seems to hold rarely, if ever, unless the sum is finite. Furthermore the derivative with 
respect to 2 is almost always nonzero.] 

53. For (b), set r = 1 and s = - I in the result of (a). 

54. Insert minus signs in a checkerboard pattern as shown. 


/ 1 -0 0 - 0 \ 

I -1 1-0 0 1 

I 1 -2 1 -0 I 

V-l 3-3 1/ 

This is equivalent to multiplying a by (-l) l+J . The result is the desired inverse, by 

Eq. (33)- 


55. Insert minus signs in one triangle, as in the previous exercise, to get the inverse of 
the other. (Eq. (47).) 


56. 210 310 320 321 410 420 421 430 431 432 510 520 521 530 531 532 540 541 542 
543 610. With a fixed, b and c run through the combinations of a things two at a time; 
with a and b fixed, c runs through the combinations of b things one at a time. 

Similarly, we could express all numbers in the form n = (“) + (3) + (“) + (f) with 
a>b>c>d>0\ the sequence begins 3210 4210 4310 4320 4321 5210 5310 5320 
We can find the combinatorial representation by a “greedy” method, first choosing the 
largest possible a, then the largest possible b for n - (“), etc. [Section 7.2. 1.3 discusses 
further properties of this representation.] 


58. [Systematisches Lehrbuch der Arithmetik (Leipzig: 1811), xxix.] Use induction 
and 

Therefore [F. Schweins, Analysis (Heidelberg: 1820), §151] the ^-generalization of (21) 


?a),(„:j/- w ”- , '=E(i)(„: t ) 


(s — n+k)k 


cri 


And the identity 1 — q l = —q t ( 1 — q 4 ) makes it easy to generalize (17) to 


The (/-normal coefficients arise in many diverse applications; see, for example, Section 
5.1.2, and the author’s note in J. Combinatorial Theory A10 (1971), 178-180. 

Useful facts: When n is a nonnegative integer, (”) is a polynomial of degree 
k(n - k) in q with nonnegative integer coefficients, and it satisfies the reflective laws 


If |q| < 1 and \x\ < 1, the q-nomial theorem holds when n is an arbitrary real number, 
if we replace the left-hand side by nfc>o((^ + Q kx )/{^ + q n+k x )). Properties of power 
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series make it, necessary to verify this only when n is a positive integer, because we can 
set q = y, the identity has then been verified for infinitely many values of y. Now we 
can negate the upper index in the g-nomial theorem, obtaining 


n 

fc >0 


(1 


■q k+r+:l x) _ 


(1 




k(k — l)/2/ r+1 


/ r+1 \k 

(~q x) = 


?(*n 


For further information, see G. Gasper and M. Rahman, Basic Hypergeometric Series 
(Cambridge Univ. Press, 1990). The g-nomial coefficients were introduced by Gauss in 
Commentationes societatis regiae scientiarum Gottingensis recentiores 1 (1808), 147- 
186; see also Cauchy [Comptes Rendus Acad. Sci. 17 (Paris, 1843), 523-531], Jacobi 
[Crelle 32 (1846), 197-204], Heine [Crellc 34 (1847), 285-328], and Section 7.2.1 4 

59 ‘ (" + !)(*) "Ur) • 


60. 


n + k — 1 
k 


This formula can be remembered easily, since it is 


n(n + 1 ) . . . (n + k — 1) 
k(k — 1 ) ... 1 

like Eq. (2) except that the numbers in the numerator go up instead of down. A 
slick way to prove it is to note that we want to count the number of integer solutions 
(ai,...,afc) to the relations 1 < ai < 0,2 < • • • < a*, < n. This is the same as 
0 < a i < a -2 + !<•••< ak + k — 1 < n + k\ and the number of solutions to 


0 < bi < 62 < • ' • < b/c < n + k 


is the number of choices of k distinct things from the set (1, 2, . . . , n + A: - 1}. (This 
trick is due to H. F. Scherk, Crelle 3 (1828), 97; curiously it was also given by W. A. 
Forstemann in the same journal, 13 (1835), 237, who said “One would almost believe 
this must have been known long ago, but I have found it nowhere, even though I have 
consulted many works in this regard.”) 

61. If a m „ is the desired quantity, we have a mn = na m(n _ 1) + S mn by (46) and 
(47). Hence the answer is [n>m]n\/m\. The same formula is also easily obtained 
by inversion of (56). 


62. Use the identity of exercise 31, with (m,n,r,s, k) +- ( m + k , l-k, m + n, n + l, j): 


/I + m\ 

/m + n\ 

f n + l\ 

\l + k ) 

\m + k ) 

\n + k) 




m — k 
l-k- j 


m + n + j\ 
J \ m + l ) 


= 21 -2j \ (m + n + j)\ 

\l-j + k)(2l- 2 j)\j\ (m - l+j)\ (n+j- 1)!’ 

by rearranging the factorial signs. The sum on k now vanishes uidess j = l. 

The case l = m = n of this identity was published by A. C. Dixon [Messenger 
of Math. 20 (1891), 79-80], who established the general case twelve years later [Proc. 
London Math. Soc. 35 (1903), 285-289]. However, L. J. Rogers had already published 
a much more general formula in the meantime [Proc. London Math. Soc. 26 (1895), 
15-32, §8], See also papers by P. A. MacMahon, Quarterly Journal of Pure and Applied 
Math. 33 (1902), 274-288, and John Dougall, Proc. Edinburgh Math. Society 25 
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(1907), 114-132. The corresponding g-nomial identities are 

y^/m-r + s\ /n + r-s\ / r + k \ {m _ r . +s _ fc)(n _ fc) = / r \ /s\ 

k \ k ) q V n — k /jlro + n// 

+ + M ( n + l \ (3fc 2 — fc)/2 _ (l + m + n)! q 

k V l + k ) q \m + k ) q \n + l\ q m\ q n\ q 

where n! q = rifc=i (! + 1 + 1“ <7 fe_1 )- 

63. See CMath, exercises 5.83 and 5.106. 

64. Let f(n,m) be the number of partitions of {1,2, ...,n} into m parts. Clearly 
/(l, m) = 5\ m . If n > 1, the partitionings are of two varieties: (a) The element 
n alone forms a set of the partition; there are f(n — 1, m — 1) ways to construct 
partitions like this, (b) The element n appears together with another element; there 
are m ways to insert n into any m-partition of { 1 , 2, . . . , n — 1}, hence there are 
mf(n — 1, m) ways to construct partitions like this. We conclude that f(n,m) = 
f(n — 1, m — 1) + mfin — 1, m), and f(n,m) = by induction. 

65. See AMM 99 (1992), 410-422. 

66. Let X = (*), X = („* x ) = X = ( n ^) = with similar notations 

for Y and Z. We may assume that y > n — 1 is fixed, so that a: is a function of z. 

Let F(z ) = X — Y — Z , and suppose that F(z) = 0 for some z > n — 2. We will 
prove that F'(z) < 0; therefore z = y must be the only root > n — 2, proving the second 

inequality. Since F(z) = (T + Z) - - z ~™ +1 Z = 0 and x > y and Y, Z > 0, 

we must have . Setting X 1 = dX/dx and Z' = dZ/dz = dX/dz, we have 

X" = 1 1 t | 1 > _n_ /l 1 \ _ n Z' 

X x x 1 i-n + 1 > n + lVz + z — n + 2 / n + 1 Z ’ 

since ^ < !=*&, . . . , < ±. Thus dx/dz = Z'/X' < =±I (Z/X), and 

F'(z) = + X -^Z' --- Z ~ n + 1 Z ' < - Z ~ n+1 )z' < 0. 

n + 1 <zz n + 1 n n Vn+1 n / 

To prove the first inequality, we may assume that n > 2. Then if X_ = Y_ + Z_ for 
some z > n — 2, the second inequality tells us that z = y. 

References: L. Lovasz, Combinatorial Problems and Exercises, Problem 13.31(a); 
R. M. Redheffer, AMM 103 (1996), 62-64. 

67. If k > 0, exercise 1.2.5-24 gives the slightly sharper (but less memorable) upper 

bounds (2) = n-/k\ < n k /k\ < ■ The corresponding lower bound is 

68. Let t k = k( n k )p k ( 1 - p) n+1 ~ k \ then t k - t k+1 = ( n k )p k { 1 - p) n ~ k (k - np). So the 
stated sum is 

^ ^ (^fc + i — tk) + ^ ^ ( tk ~ tk+i) — 2t[- np i . 

fc<fnpl fc>T n Pl 

[De Moivre stated this identity in Miscellanea Analytica (1730), 101, in the case that 
np is an integer; H. Poincare proved the general case in his Calcul des Probability 
(1896), 56-60. See P. Diaconis and S. Zabell, Statistical Science 6 (1991), 284-302, for 
the interesting history of this identity and for a variety of similar formulas.] 


i 
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SECTION 1.2.7 

1. 0, 1, and 3/2. 

2. Replace each term l/(2 m + k) by the upper bound l/2 m . 

3. H^2_ l < Eo<Km 2 fc /2 fer ; 2' 1 /(2 r_1 — 1) is an upper bound. 

4. (b) and (c). 

5. 9.78760 60360 44382 . . . 

6. Induction and Eq. 1.2.6-(46). 

7. T(m + 1, n) — T(m, n) = l/(m+ 1) — l/(mn + 1) — • • • — 1 /(mn + n ) < l/(m + 1) — 
(1/ (mn + «) + ••• + 1 /{mn + n)) = 1 /(m + 1) - n/(mn + n) = 0. The maximum 
value occurs at m = n = 1, and the minimum is approached when m and n get very 
large. By Eq. (3) the greatest lower bound is 7, which is never actually attained. A 
generalization of this result appears in AMM 70 (1963), 575-577. 

8. By Stirling’s approximation, Inn! is approximately (ra + ±) Inn - n + In i/2tv; also 
ELi H k is approximately (n+1) lnn-n(l-7) + ( 7 +±); the difference is approximately 
7 n + 4 Inn + .158. 

9. — 1/n. 

10. Break the left side into two sums; change k to k + 1 in the second sum. 

11. 2 — H„/n — 1/n, for n > 0. 

12. 1.000. . . is correct to more than three hundred decimal places. 

13. Use induction as in the proof of Theorem A. Or use calculus: Differentiate with 
respect to x, also evaluate at x = 1. 

14. See Section 1.2.3, Example 2. The second sum is §(tf 2 +1 - H^). 

15- £” =1 (l/j) Ek=j H k can be summed by formulas in the text; the answer comes to 
(n + 1 )H n — (2 n T 1 )H n -f- 2n. 

16. H 2n -i - \H n -i. 

17. First solution (elementary): Taking the denominator to be (p - 1)!, which is a 
multiple of the true denominator but not a multiple of p, we must show only that the 
corresponding numerator, (p - 1)!/1 + (p - l)!/2 + ••• + (?- l)!/(p - 1), is a multiple 
of p. Modulo p, (p - 1)!/A: = (p - 1)! k', where k' can be determined by the relation 
kk' mod p = 1. The set {T, 2 (p - 1)'} is just the set {1, 2, . . . , p - 1 }; so the 
numerator is congruent to (p — 1)! (1 + 2 + • • • + p — 1) = 0. 

Second solution (advanced): By exercise 4.6.2-6, we have = x p - x (modulo p); 
hence [[(] = 6 kp - Ski, by exercise 1.2.6-32. Now apply exercise 6. 

The numerator of H p - 1 is in fact known to be a multiple of p 2 when p > 3; see 
Hardy and Wright, An Introduction to the Theory of Numbers, Section 7.8. 

18. If n = 2 k m where m is odd, the sum equals 2 2k m 1 /m 2 where rni and m 2 are both 
odd. [AMM 67 (1960), 924-925.] 

19. Only rt == 0, n = 1. For n > 2, let k = [lgnj. There is precisely one term whose 
denominator is 2 k , so 2 k 1 H„ — | is a sum of terms involving only odd primes in the 
denominator. If H n were an integer, 2 k 1 H n — | would have a denominator equal to 2. 

20. Expand the integrand term by term. See also AMM 69 (1962), 239, and an article 
by H. W. Gould, Mathematics Magazine 34 (1961), 317-321. 

21. Hl +l - H^ v 
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22. (n + 1 )(H 2 n - H {2) ) - 2 n(H n - 1). 

23. r'(n + 1)/F(n + 1) = 1 jn + r'(n)/T(n), since T(a: + 1) = xT(x). Hence H n = 
7 + T'(n + l)/r(n + 1). The function ip(x) = T' (x)/T(x) = H x - 1 — 7 is called the psi 
function or the digamma function. Some values for rational x appear in Appendix A. 

24. It is 


x lim e (H ^ Xun)x 


ri(( i+ ik"‘) 


.. x(x + 1) . . . (x + n) 
lim — p 

n-* 00 n x n\ 


Note: The generalization of H„ considered in the previous exercise is therefore equal 
to = £ fc> 0 ( 1 /(fc + l) r — l/(fc + 1 + *) r ) 5 when r = 1; the same idea can be used 
for larger values of r. The infinite product converges for all complex x. 


SECTION 1.2.8 

1. After k months there are Fk + 2 pairs, so the answer is F14 = 377 pairs. 

2. ln(i/> 1000 /v / 5 ) = 1000 In <£- |ln5 = 480.40711; log 10 F1000 is l/(lnl0) times this, 
or 208.64; F1000 is therefore a 209-digit number whose leading digit is 4. 

4. 0, 1, 5; afterwards F n increases too fast. 

5. 0, 1, 12. 

6. Induction. (The equation holds for negative n also; see exercise 8.) 

7. If d is a proper divisor of n, Fa divides F n . Now Fa is greater than one and less 
than F n provided d is greater than 2. The only nonprime number that has no proper 
factor greater than 2 is n = 4; F4 = 3 is the only exception. 

8. F_ 1 = 1; F —2 = —1; F- n = ( — l) n+1 F„ by induction on n. 

9. Not (15). The others are valid, by an inductive argument that proves something 
true for n — 1 assuming it true for n and greater. 

10. When n is even, it is greater; when n is odd, it is less. (See Eq. (14).) 

11. Induction; see exercise 9. This is a special case of exercise 13(a). 

12. If g(z) = £ F n z n , (1 - 2 - z 2 )g(z ) =z + Foz 2 + F lZ 3 + ■ ■ ■ = z + z 2 G{z). Hence 
g(z) = G(z) + zG(z) 2 ; from Eq. (17) we find F„ = ((3 n + 3)/5 )F n — ( n/5)F„+i . 

13. (a) a n = rF n -i + sF n . (b) Since ( b n+2 + c) = (6 n +i + c) + (6„ + c), we may 
consider the new sequence b' n = b n + c. Applying part (a) to b' n , we obtain the answer 
cF n - 1 + (c + l)F n - c. 

_ , _ _ /n\ ( n + 1 \ / n + 

14. a„ - F m+n+1 +F n — ( m ) - ( m _ J ( Q )• 

15. c n = xa n + yb n + (1 — x — y)F n . 

16. F n+1 . Induction, and (" + \ ~ k ) = (" ^ * ) + ( ( " “ if 

17. In general, the quantity ( x n+k - y n+k )(: v m ~ k - y m ~ k ) - {x n - y n )(x m - y m ) is 
equal to (xy) n (x m ~ n ~ k — y m ~ n ~ k )(x k — y k ). Set x = <f>, y = and divide by (\/5) 2 . 

18. It is l‘‘ 2 n - 1 - 

19. Let u = cos 72°, v = cos 36°. We have u = 2v 2 — 1; v = 1 — 2sin 2 18° = 1 — 2 u 2 . 

Hence u + v = 2(v 2 — u 2 ), i.e., 1 = 2(v — u) = 2v — 4v 2 + 2. We conclude that 
v = | <j>. (Also u = sin 36° = ^ 1//2 , sin 72° = Another 

interesting angle is a = arctan^ = f + f arctan |, for which we have sina = 5 _1 ' 4 <k , 
cosa = 5 _1 ^ 4 (/> -1 ^ 2 .) 
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20 . F n+ 2 — 1 . 

21. Multiply by x 2 + x-l; the solution is (x n+1 F n+1 + x n+2 F n - x) / (x 2 + x - 1) . If the 
denominator is zero, x is 1/0 or 1/f; then the solution is (n + 1 - x n F n+x )/(2x + 1). 

22. F m+ 2 n', set t = 2 in the next exercise. 

23 ‘ - ^ k F t k F t n _-U m ) 

= + F ‘-i) n - + Ft-O") = F m+tn . 

24 . F n +i (expand by cofactors in the first row). 

25 . 2 n v / 5F n = (1 + V5) n - (1 - y/E) n . 

26. By Fermat s theorem, 2 P - 1; now apply the previous exercise and exercise 

1.2.6-10(b). 

27 . The statement is true if p = 2. Otherwise F p _iF p+1 - F 2 = -1; hence, from the 
previous exercise and Fermat’s theorem, F P - X F P+ 1 = 0 (modulo p). Only one of these 
factors can be a multiple of p, since F p+ i = F p + F p - X . 

28 . <f> n . Note: The solution to the recurrence a n +i = Aa„ + B n , a 0 = 0, is 


an = (A n - B n )/(A - B) if A / B, o n = nA n ~ 1 if A = B. 


29 - to Or Or Or Or Or Or Or 

1 0 0 0 0 0 0 

1 1 0 0 0 0 0 

1 1 1 0 0 0 0 

1 2 2 1 0 0 0 

1 3 6 3 1 0 0 

1 5 15 15 5 1 0 

1 8 40 60 40 8 1 

(b) follows from (6). [E. Lucas, Amer. J. Math. 1 (1878), 201-204.] 


30. We argue by induction on m, the statement being obvious when m= 1: 

o>) 

= (-!) m ^ E( m fc '^(- l )^- 1 -^ 21 ^- 2 = 0 . 

(c) Since ( — 1) Fm-fc = Fk- X F rn — Fk F m _ i and F m / 0, we conclude from (a) 
and (b) that 0) T (~O (m ~ k)/A K+k F k-x = 0 . 

(d) Since F n+ k = Fk~ x F n + FkF n+x the result follows from (a) and (c). This 
result may also be proved in slightly more general form by using the g-nomial theorem 
of exercise 1.2.6-58. References: Dov Jarden, Recurring Sequences, 2nd ed. (Jerusalem. 
1966), 30-33; J. Riordan, Duke Math. J. 29 (1962), 5-12. 


31. Use exercises 8 and 11. 

32. Modulo F n the Fibonacci sequence is 0, 1, ... , F„_i, 0, F n - 1 , — F n _ 2 , 

33. Note that cosz = | (e lz + e~ lz ) = —i/2, for this particular z; then use the fact 
that sin(n + l)z + sin(n — l)z — 2 sin nz cos z, for all z. 
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34 . Prove that the only possible value for F kl is the largest Fibonacci number less 
than or equal to n; hence n — F kl is less than F kl ~ i, and by induction there is a unique 
representation of n — F kl ■ The outline of this proof is quite similar to the proof of the 
unique factorization theorem. The Fibonacci number system is due to E. Zeckendorf 
[see Simon Stevin 29 (1952), 190-195; Bull. Soc. Royale des Sciences de Liege 41 (1972), 
179-182]; but Section 7. 2. 1.7 points out that it was implicitly known in 14th-century 
India. Generalizations are discussed in exercise 5.4.2-10 and in Section 7.1.3. 

35 . See G. M. Bergman, Mathematics Magazine 31 (1957), 98-110. To represent x > 0, 
find the largest k with <j> k < x and represent x as <j> k plus the representation of x — <f> k . 

The representation of nonnegative integers can also be obtained from the following 
all-integer recursive rules, starting with the trivial representations of 0 and 1: Let 
L n = 4> n + <f> n = F n+ 1 + F n - 1 . The representation of L 2n + m for 0 < m < L 2n - 1 and 
n > 1 is 4> 2n + <t>~ 2n plus the representation of m. The representation of L 2n +i + rn for 
0 < m < L 2n and n > 0 is (f> 2n+1 + f>~ 2n ~ 2 plus the representation of m — <j>~ 2n , where 
the latter is obtained by applying the rule (j> k — </> fc_2j = <f> k ~ 1 + cj > k ~ 3 + • . . + ( j > k ~ 2 i+ 1 , 
It turns out that all strings a of Os and Is, such that a begins with 1 and has no 
adjacent Is, occur to the left of the radix point in the representation of exactly one 
positive integer, except for the strings that end with 10 2fc l; the latter strings never 
occur in such representations. 

36 . We may consider the infinite string Soo, since S„ for n > 1 consists of the first F n 
letters of S^,. There are no double a’s, no triple 6’s. The string S n contains F n _ 2 a’ s 
and F n —\ 6’s. If we express m — 1 in the Fibonacci number system as in exercise 34, 
the mth letter of Soo is a if and only if k T = 2. The fcth letter of Soo is 6 if and 
only if [(fc + 1)</> _1 J - [fc^' 1 ] = 1; the number of 6’s in the first k letters is therefore 
[(fc + 1)0 -1 J. Also, the fcth letter is 6 if and only if k = [m<f ) J for some positive 
integer m. This sequence was studied by John Bernoulli III in the 18th century, by 
A. A. Markov in the 19th, and by many other mathematicians since then; see K. B. 
Stolarsky, Canadian Math. Bull. 19 (1976), 473-482. 

37 . [ Fibonacci Quart. 1 (December 1963), 9-12.] Consider the Fibonacci number 

system of exercise 34; if n = F kl 4 h F kr > 0 in that system, let /r(n) = F kr . Let 

/r(0) = oo. We find that: (A) If n > 0, fi{n - fr(n)) > 2/r(n). Proof: /j.(n - fr(n)) = 
Fk r _ i > F kr+ 2 > 2 F kr since k r > 2. (B) If 0 < m < F k , y,{m) < 2 (F k —m). Proof: Let 
fj.(m) = Fj- m < F k -i + F k _ 3 + ■ ■ • + F j+ ^ k _ 1 _ j - ) mod 2 = mod 2 + F k < 

— \Fj+F k . (C) If 0 < m < n(n), fr{n) + m) < 2(n(n) — rn). Proof: This follows 

from (B). (D) If 0 < m < /r(n), n(n — m) < 2m. Proof: Set m = fi{n) — m in (C). 

Now we will prove that if there are n chips, and if at most q may be taken in the 
next turn, there is a winning move if and only if /a(n) < q. Proof: (a) If /r(n) > q all 
moves leave a position n', q with n(n') < q. [This follows from (D), above.] (b) If 
/r(n) < q, we can either win on this move (if q > n) or we can make a move that leaves 
a position n, q with ^ u(n , ) > q. [This follows from (A) above: Our move is to take 
n(n) chips.] It can be seen that the set of all winning moves, if n = F kl + • • • + F kr , 
is to remove F kj + • • • + F kr , for some j with 1 < j < r, provided that j = 1 or 
■Pfcj-i > 2(Ffcj + • • • + F kr ). 

The Fibonacci representation of 1000 is 987+13; the only lucky move to force a vic- 
tory is to take 13 chips. The first player can always win unless n is a Fibonacci number. 

The solution to considerably more general games of this type has been obtained 
by A. Schwenk [Fibonacci Quarterly 8 (1970), 225-234]. 

39 . (3 n — ( — 2) n )/5. 
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40. We prove, by induction on m, that f(n) = m for F m < n < F m+ 1: First, f(n) < 
max(l + f(F m ), 2 + f(n- F m )) = m. Second, if /(n) < m there is some k < n with 
! + /(&) < m (hence k < F m - 1) and 2 + f(n — k) < m (hence n — k < Fm- 2 ); but then 
n < F m - 1 + F m - 2- [Thus the Fibonacci trees defined in Section 6.2.1 minimize the 
maximum root-to-leaf cost when a right branch costs twice as much as a left branch.] 

41. Fk 1 + 1 + • • • + Fk r + 1 = 4>n + ( <!> kl H 1- <fi kr ) is an integer, and the parenthesized 

quantity lies between f 5 H = </ _1 - 1 and $ 2 + fi 4 H = 0 -1 . Similarly, 

Tfcj _i + ■ • • + F kr - 1 = 4> 1 n + (^ fcl + • • • + $ kr ) = /(</> x n). [Such Fibonacci shifting is 
a convenient way to convert mentally between miles and kilometers; see CMath , §6.6.] 

42. [Fibonacci Quarterly 6 (1968), 235-244.] If such a representation exists, we have 

mF/v-i + uFn = F^+n + Fk 2 +N + • • • + Fk r +N (*) 

for all integers JV; hence two different representations would contradict exercise 34. 

Conversely, we can prove the existence of such joint representations for all non- 
negative m and n by induction. But it is more interesting to use the previous exercise, 
and to prove that such joint representations exist for possibly negative integers m and n 
if and only if m + 4>n > 0: Let N be large enough so that 4- nc^ N \ < 

and represent mF/v- 1 + nF N as in (*). Then mF N + nF N+l = <p(mF N - x + nF N ) + 
+ n<fi N ) = f(4>(mF N - 1 + nF N )) = F kl+ N+1 + • • • + F kr+N+1 , and it follows 
that (*) holds for all TV. Now set TV = 0 and TV = 1. 


SECTION 1.2.9 

1. 1/(1 - 2z) + 1/(1 - 3z). 

2. It follows from (6), since (£) = n\/k\(n - k)\. 

3. G'(z) = ln(l/(l — z))/( 1 — z) 2 + 1/(1 — z) 2 . From this and the significance of 

G(z)/{ 1 — z), we have H k = nH n — n; this agrees with Eq. 1.2.7— (8). 

4. Put t = 0. 

5. The coefficient of z k is, by (n) and (22), 

(rc ~ 1)! ^ / j \ (k\ 
k ' 


Now apply Eqs. 1.2.6-(46) and 1.2.6-(52). (Or, differentiate and use 1.2.6-(46).) 

6. (ln(l/(l - z))) 2 ; the derivative is twice the generating function for the harmonic 
numbers; the sum is therefore 2H n -i/n. 


8- 1/((1 — z )(l - z 2 )( 1 — z 3 ) . . . ). [This is historically one of the first applications 
of generating functions. For an interesting account of L. Euler’s eighteenth-century 
researches concerning this generating function, see G. Polya, Induction and Analogy in 
Mathematics (Princeton: Princeton University Press, 1954), Chapter 6.] 

9. ±Si + + |S 2 2 + | SrSa + \S 4 . 


10. G(z ) = (1 + xiz) . . . (1 + x n z). Taking logarithms as in the derivation of Eq. (38), 
we have the same formulas except that (24) replaces (17), and the answer is exactly the 

same except that S 2 , S 4 , S 6 , . . . are replaced by -S 2 , -S 4 , -S 6 , We have ei = Si, 

e 2 = - |5 2 , e 3 = - ISx5 2 + |5 3 , e 4 = - \S 2 S 2 + |S 2 2 + |5r5 3 - \S 4 . 

(See exercise 9.) The recurrence analogous to (39) is ne n = 5ie n _i - S 2 e n - 2 + ■ ■ ■ . 
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Note: The equations in this recurrence are called Newton’s identities , since they were 
first published in Isaac Newton’s Arithmetics Universalis (1707); see D. J. Struik’s 
Source Book in Mathematics (Harvard University Press, 1969), 94-95. 

11. Since £ m>1 SmZ m /m = lnG(z) = £ fc>1 ( — I) k ~ l (h\z + h 2 Z 2 + ■ • • ) k /k, the desired 

coefficient is (~l) kl+k2+ ' +km ~ 1 in(ki + k 2 \-k m - l)!//c x ! k 2 \ . . . k m \. [Multiply by 

( — l) m 1 to get the coefficient of e kl e k2 . . . e£p when S m is expressed in terms of the 
e’s of exercise 10. Albert Girard stated the formulas for Si, S 2 , S 3 , and S 4 in terms 
of e.\ , e 2 , e3, and e 4 near the end of his Invention Nouvelle en Algebre (Amsterdam: 
1629); this was the birth of the theory of symmetric functions.] 


12. Yi a mnW rn Z 1 = Y (”) U/ ’ V 

m,n> 0 m,n> 0 


^(l + w)V 

n>0 


1/(1 -z-wz). 


13. f n ‘ +1 e st f(t) dt = (ao + ■ ■ ■ + a n )(e sn — e 3 ( n+1 ))/s. Adding these expressions 
together for all n, we find L/(s) = G(e~ s )/s. 


14. See exercise 1.2.6-38. 


15. G n (z) — Gn_i(z) + zGn— 2 (z) + S„o, so we find H{w) = 1/(1 — w — zw 2 ). Hence, 
ultimately, we find 


C „ W= ((l±4±T)"«-(lz4±T)-) / V TT4I 


G„(- 1) = (n + l)/2 n for n > 0. 

16. G nr (z ) = (1 + z + • • • + z r ) n = ^ ^ . [Note the case r = oo.] 


"■ E(T)m‘ = E 


w(w + 1 ) . . . (w + k — 1 ) 


fc(fc -!)...! -^=E[ n j^ n A!- 


(Alternatively, write it as e TOln (i/(i A) an( j expanc i fi rs t by powers of w.) 

18. (a) For fixed n and varying r, the generating function is 

G n (z) = (1 + z)( 1 + 2z) . . . (1 + nz) = z n+1 (-) Q + X ) ( \ + 2 ) • • • ( \ + n) 


E 


n + 1 
k 


n+l — k 


by Eq. (27). Hence the answer is ■ (b) Similarly, the generating function is 

1 1 1 _ f fc 1 k-n 

1—z 1 — 2 z 1 — nz nj 

k 

by Eq. (28), so the answer is { n ^ r }. 

19- £n>i (l/« - V(n + P /q))x p+nq = ZVo «' ~ k ” ln(l - u k x) - x p ln(l - a*) + = 

f(x) + g(x), where w = e 2l ' l ^ q and 

q— 1 

f(x) = Y W ~ kp ln(l - u k x), g(x) = (1 - x p ) ln(l - *) + V - x p In 

f— ; p 1 - x 
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Now lim^-n- g(x) = q/p — lng. Prom the identity 

ln(l - e ie ) = In ^2e i(e_,r)/2 e ,e/2 -e~ l9/2 ^ 

we may write lim*_n_ f(x) = f( 1) = A + B where 
<?-l 


= In 2 + \i{6 — 7r) + In sin ^ , 


A = ^uT fcp (ln2- 


k=l 

q-1 


in ikn\ 

T + t) 


• In 2 + — + — ; 

2 + («-*>-!)’ 


B = XI w ^lnsin-Ti- = ^ (w * p + cu (<i k)p ) In sin 'fn 


0<fc<g/2 


2 pfc 1 • 

— 2 > cos 7r-lnsm— n. 

0 <k<q / 2 « 1 


Finally, 


i_ i - i fl + ojP \ _ if uj p/2 +uj p/2 \ 1 p 

2 (w-p-1) 2V1-o;pJ _ 2 V^p/2 -u-p/2j ~ ~2 COt ~q*' 

[Gauss derived these results in §33 of his monograph on hypergeometric series, Eq. [75], 
but with insufficient proof; Abel provided a justification in Crelle 1 (1826), 314-315.] 

20. c m k = by Eq. 1.2.6-(45). 

21. We find z G ( z ) + zG(z ) = G(z) — 1. The solution to this differential equation is 

G( z ) = (—l/z)e 1/z (Ei(— 1/z) + C), where £ 1 ( 2 ) = e~ { dt/t and C is a constant. 

This function is very ill-behaved in the neighborhood of 2 = 0, and G(z) has no power 
series expansion. Indeed, since \/nT ~ nje is not bounded, the generating function 
does not converge in this case; it is, however, an asymptotic expansion for the stated 
function, when 2 < 0. [See K. Knopp, Infinite Sequences and Series (Dover, 1956), 
Section 66.] 

22. G(z ) = (1 + 2 ) (1 + 2 2 ) r (l + 2 4 ) r (l + z s ) r ... = (1 — z)~ r . It follows that the 
stated sum is ( r+P_1 ). 

23. (a) When m = 1 this is the binomial theorem, with fi(z) = z and gi(z) = 1 + 2 . 
When m > 1 we can increase m by 1 if we replace z m by 2 m (l + 2~Vi) and let 
fm+l(zi, ...,Zm+l) = Z rn+ if rn (z 1 ,...,Zm-l,Z m (l + Z 7n 1 +1 )), g m + l{zi, . . . , Z m +l) = 

Zm+lQm, \Z\ j • • • > Zm — 1 5 Zxn (1 + Z'm- f -1 )). Thus 92(21 , Z2) = Zi + Z2 + Z1Z2 and 

9m ( Z\ , . . . , Zm) ^ ^ Z j 

/m(^lr • • j 2 m ) —1 

i + ^ 

1+ 


1 + z m 

Both polynomials / m and g m satisfy the same recurrence f m = z m fm-i + Zm-i/m-2, 
9m = z m gm-i +2 m -i9m-2, with the initial conditions /_i = 0, /o = g ~ i = g 0 = zo = 1. 
It follows that g m is the sum of all terms obtainable by starting with 21 . . . z m and 
striking out zero or more nonadjacent factors; there are F m+2 ways to do this. A 
similar interpretation applies to f m , except that 21 must remain. In part (b) we will 
encounter the polynomial h m = z m gm - 1 + 2 m-i/m- 2 ; this is the sum of all terms 
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obtained from z x . . . z m by striking out factors that are not cyclically adjacent. For 
example, h 3 = Z1Z2Z3 + ziz 2 + Z1Z3 + z 2 z 3 . 

(b) By part (a), S n (g lf . . . , = [<] YZ**? *ZT T KT r 9 r m \ hence 

S n ( Zl ,...,z m ) = Y, ( K ’ s ){ n ~ r )a r - s b s c s d n - r - s , 


where a — z m <; m -i, b — z m -i£lm-2, c — z m fm- 1, d = Zm-ifm- 2 - Multiplying this 
equation by z n and summing first on n, then on r, then on s, yields the closed form 


S n (zi, . . . , Zm) = [ Z n ] 


p n+l _ a n+ 1 


(1 — az)(l — dz) — bcz 2 


where 1 — (a + d)z + (ad — bc)z 2 = (1 — pz)( 1 — az). Here a + d = h m , and ad — be 
simplifies to ( — l) rn zi . . . z m . [We have, incidentally, established the recurrence S n = 
h m S n - 1 — ( — l) m zi . . . z m S n - 2, a relation that is not easy to derive without the help 
of generating functions.] 

(c) Let pi = (z + V z 2 + 4z )/2 and ai = (z — \/z 2 + 4z )/2 be the roots when 
to = 1; then p m = pj" and a m = a™. 

Carlitz used this result to deduce a surprising fact: The characteristic polynomial 
det(x7 — A) of the n x n matrix 

( 0 0 ... 0 0 \ 

A= 0 ° ••• 9 o 

V(V) rr 1 ) ••• c=j) Czl)J 

of “right justified binomial coefficients” is J2k ik)r ( _1 ) r(rl_k, / 2 l x k , with Fibonomial 
coefficients (see exercise 1.2.8-30). He also showed, using similar methods, that 



[Collectanea Math. 27 (1965), 281-296.] 

24 . Both sides are equal to J2 k (T) [ zn ]{ z G(z)) k . When G(z) = 1/(1 - z), the identity 

becomes (”)(": l) - ("*+»- 1 ) j a case of 1.2.6-(ai). When G(z) = (e z - l)/z, it 
becomes = m "> E 9- 1.2.6-(45). 

25 . £>1(1-2™)" [*»] z k (l+z) 2n - 2k = [z n ] (1 + z) 2 " J2 k [w k ] (1 — 2w)"(z/(l + z) 2 ) fc , 
which equals [ z" ] (1 + z) 2n (l - 2z/(l + z) 2 ) n = [z n ] (1 + z 2 )" = ( n " 2 )[n even]. Sim- 
ilarly, we find £ fc ([!) ( 2 £fc fc ) (— 4) fc = (— l)"( 2 r [ i ). Many examples of this summation 
technique can be found in G. P. Egorychev’s book Integral Representation and the 
Computation of Combinatorial Sums (Amer. Math. Soc., 1984), translated from the 
Russian edition of 1977. 

26 . (F(z)[ G(z) denotes the constant term of F(z~ 1 )G(z). See the discussion by D. E. 
Knuth in A Classical Mind (Prentice-Hall, 1994), 247-258. 
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SECTION 1.2.10 

1. G„(0) = 1/n; this is the probability that X[n] is the largest. 

2 - G"( 1) = J2 k - 1 )Pk, G'( 1) = k Pk . 

3. (min 0, ave 6.49, max 999, dev 2.42). Note that H i 2 ^ is approximately 7r 2 /6; see 

Eq. 1.2.7-(r). 

4. (l)p k q n - k . 

5. The mean is 36/5 = 7.2; the standard deviation is 6 a/2/5 « 1.697. 

6. For (18), the formula 

ln( q+pe) = In (l + pt + ~ + ■ ■ • j = pt + p(l - p) +p(l - p)(l - 2p) j + • • • 

tells us that n 3 /n = p(l — p)(l — 2 p) = pq(q — p). (This nice pattern does not continue 
to the coefficient of t 4 .) Setting p = AT 1 gives us k 3 = fc _1 (l - A- _1 )(l - 2 A -1 ) = 

H n - + 2 Hn 3) in the case of distribution (8). And for (20), we have lnG(e ( ) = 

t + H(nt) — H (t) where H(t) = ln((e‘ - 1 )/t). Since H'(t) = e*/( e * - 1) - 1/t, we have 
K r = ( n r — 1 )B r /r for all r > 2 in this case; in particular, k 3 = 0. 

7. The probability that A = k is p m k- For we may consider the values to be 

I, 2, . . . , m. Given any partitioning of the n positions into m disjoint sets, there are m\ 
ways to assign the numbers 1, . . . , m to these sets. Algorithm M treats these values as 
if only the rightmost element of each set were present; so p mk is the average for any 
fixed partitioning. For example, if n = 5, m = 3, one partition is 

{X[1],X[4]} {X[2],X[5]} {X[3]} ; 

the arrangements possible are 12312, 13213, 21321, 23123, 31231, 32132. In every 
partition we get the same percentage of arrangements with A = k. 

On the other hand, the probability distribution does change if more information 
is given. If n = 3 and m = 2, for example, our argument in the previous paragraph 
considers the six possibilities 122, 212, 221, 211, 121, 112; if we know that there are 
two 2s and one 1, then only the first three of these possibilities should be considered. 
But this interpretation is not consistent with the statement of the exercise. 

8. M-fM n . The larger M is, the closer this probability gets to one. 

9. Let q nm be the probability that exactly m distinct values occur; then from the 
recurrence 

_ M — m + 1 m 

9nm ~ M <?(n — l)(ro — 1) + -^<7(n-l)m 

we deduce that 

qnm = M! { ” }/(M-m)!M n . 

See also exercise 1.2.6-64. 

10. This is qnmPmk summed over all m, namely M~ n (m){m}L+ 1]- There does 
not appear to be a simple formula for the average, which is one less than 

«->' - 1 (> - s)"™- =«.+£((;)- A 

II. Since this is a product, we add the semi-invariants of each term. If H(z) = z n , 
H{e *) = e nt , so we find Kr = n and all others are zero. Therefore, mean(F) = n + 
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mean(G), and all other semi-invariants are unchanged. (This accounts for the name 
“semi-invariant.” ) 


12. The first identity is obvious by writing out the power series for e kt . For the second, 
let u = 1 + Mit + M 2 t 2 /2\ + • • • ; when t = 0 we have u = 1 and D k u = M k ■ Also, 
(In u) = (-1) J-1 0' — l)!/tP. By exercise 11, the same formula applies for central 
moments except that we leave out all terms with fci > 0; thus k 2 = m 2 , «3 = m3, 
«4 = m 4 — 3m 2 . 


“■ G " w 

Let z n = e lt ^ n . When n — > 00 and t is fixed, we have z n — > 1; hence F(z n + 1) — > 1, 
and 

-itp n 


lim z n >ln G n (z n ) = lim exp 


= lim exp 


+ ( c «/*» _ i)lnn 


-t 2 In n 

2oi 


+ 0 


%/log? 


-tv 2 
= e ' . 


Notes: This is a theorem of Goncharov [Izv. Akad. Nauk SSSR Ser. Math. 8 (1944), 
3-48]. P. Flajolet and M. Soria [Disc. Math. 114 (1993), 159-180] have extended the 
analysis to show that G n (z ) and a large family of related distributions not only are 
approximately normal near their mean values, they also have uniformly exponential 
tails, in the sense that 


Pr 


AT 


/^n 



< e 


— ax 


for some positive constant a and for all n and x. 

14. e-^/^^+pe"/^)" = (ge-^/v^+pe^/v^)". Expand the exponen- 
tials in power series, to get (l-t 2 /2n + 0(n~ 3/2 )) n = exp(n ln(l - t 2 /2n + 0(n~ 3/2 ))) = 
exp(— f 2 /2 + 0(n _1/2 )) e xp(-f 2 /2). 

15. (a) J2k>o e M (/ Lt2 )V^- = (b) lne^ e = /u(e t — 1), so all semi-invariants 

equal p. (c) exp (-itnp/^/np) exp(np(it / ^/rvp - t 2 /(2np) + 0(n _3/2 ))) = exp(-£ 2 /2 + 
G(n - 1 / 2 )). 


16. g(z) = YjkP* 9 k{z)\ mean(p) = J^ k Pk mean(p fc ); and var(p) = ^2 k p k vai(g k ) + 
Sj<fcPiPfc(mean(flj-) - mean(p fc )) 2 . 

17. (a) The coefficients of f(z) and g(z) are nonnegative, and /( 1) = p(l) = 1. Clearly 
h(z) shares these same characteristics, since h( 1) = g(f( 1)), and the coefficients of h 
are polynomials in those of / and g , with nonnegative coefficients. 

(b) Let f(z) = Y,PkZ k where p k is the probability that some event yields a “score” 
of k. Let g(z) = J2lkZ k where q k is the probability that the event described by / 
happens exactly k times (each occurrence of the event being independent of the others). 
Then h(z) = r k z k , where r k is the probability that the sum of the scores of the events 
that occurred is equal to k. (This is easy to see if we observe that f{z) k = ^2s t z t , 
where st is the probability that a total score t is obtained in k independent occurrences 
of the event.) Example: If / gives the probabilities that a woman has k female offspring, 
and if g gives the probabilities that there are k females in the nth generation, then h 
gives the probabilities that there are k females in the ( n + l)st generation, assuming 
independence. 

(c) mean(h) = mean(p) mean(/); var(h) = var(p) mean 2 (/) + mean(g) var (/). 
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18. Consider the choice of X [1], . . . , X[n] as a process in which we first place all the 

n ’ s > then Pl ace all the (n - l)’s among these n’s, ..., finally place the ones among 
the rest. As we place the r’s among the numbers {r + 1,. ,,,n}, the number of local 
maxima from right to left increases by one if and only if we put an r at the extreme 
right. This happens with probability k T /(k r + fc r +i H f k n ). 

19. Let ii !,■ /. Then is a left-to-right maximum of a i . . . cL n < j c k implies CLj <C l 

<=> a.j > l implies j > k 4=4> j > l implies bj > k <=^ k is a right-to-left minimum of 
b i . . . b n . 

20. We have m L = maxjai - 6i,...,a„ - b,,}. Proof: If not, let k be the smallest 
subscript such that a k — b k > m l ■ Then ak is not a left-to-right maximum, so there is 
& j < k with aj > a k ■ But then aj — bj > a k — b k > m.L , contradicting the minimality 
of k. Similarly, m R = max{6i-oi, . . ,,b n -a n }. 

21. The result is trivial when e > q, so we may assume that e < q. Setting x = £±S -3- c 
in (25) gives Pr(X > n(p + e)) < ((^) p+£ (^) 9 - { ) n . Now (^f +£ < e'^since 

t < e ( 1 for all real t. And (q - e) In ^ = c - i e 2 q ^ - ^e 3 q~ 2 < e - X € 2 . 

(A more detailed analysis yields the slightly stronger estimate exp( — e 2 n/(2pq)) when 
P > §; st iH further work yields the upper bound exp(-2e 2 n) for all p.) 

By reversing the roles of heads and tails we find 

Pr(X < n(p - e)) = Pr(n - X > n{q + e)) < e -' 2,l/(2p) . 

(One should not confuse “tails” with the tail of a probability distribution.) 

22. (a) Set x = r in (24) and (25), and note that q k + p k r = 1 + (r — 1 )p k < e (r ^ 1)pt , 
[See H. Chernoff, Annals of Math. St at. 23 (1952), 493-507.] 

(b) Let r = 1 + 5 where |<5| < 1. Then r^e^ 1 = exp(-^S 2 + ^S 3 -■■•), which 
is < e~ s ! 2 when 5 < 0 and < e~ s2/3 when <5 > 0. 

(c) The function r~ 1 e 1 ~ r decreases from 1 to 0 as r increases from 1 to 00. If 
r > 2 it is < |e 1/2 < .825; if r > 4.32 it is < §. 

r Incidentally, the tail inequalities with x = r give precisely the same estimate 
(r r e r 1 ) p when A has the Poisson distribution of exercise 15. 

23 . ^ Setting 2: = in (24) gives Pr(X < n(p - e)) < (( < 

e - e n/(2 pg ) Similarly, x = yields Pr(A > n(p + e)) < (( 5 P -)P+«(2±£)9+£)n_ 

Let /(e) = (q + e) ln(l+ |) - (p + e) ln(l + 1 ), and note that /'(e) = ln(l + 1 ) - ln(l + -). 
It follows that /(e) < -e 2 /(6 pq) if 0 < e < p. 

SECTION 1.2.11.1 

1. Zero. 

2. Each O-symbol represents a different approximate quantity; since the left-hand 
side might be f(n) - (-/(n)) = 2/(n), the best we can say is 0(/(n)) - 0(/(n)) = 
0(/(n)), which follows from (6) and (7). To prove (7), note that if |x„| < M|/(n)| for 
n > n 0 and |®^| < M'|/(n)| for n > no, then |x n ±*^| < \x n \ + \x' n \ < (M + M')|/(n)| 
for n > max(no,n[j). (Signed, J. H. Quick, student.) 

3. n(lnn) + yn + 0(y/n\nn). 

4. In a + (lna) 2 /2n + (lna) 3 /6n 2 + 0(n -3 ). 

5. If /(n) = n 2 and g(n) = 1, then n belongs to the set 0(/(n) + g(n)) but not to 
the set f(n) + 0(g(n)). So the statement is false. 


1 . 2 . 11.2 


ANSWERS TO EXERCISES 503 


6 . A variable number, n, of O-symbols has been replaced by a single O-symbol, falsely 
implying that a single value of M will suffice for each term \kn\ < Mn. The given sum is 
actually 0(n 3 ), as we know. The last equality, X^=i O(n) = 0(n 2 ), is perfectly valid. 

7. If x is positive, the power series 1.2. £>-( 22 ) tells us that e x > x m+1 /(m+ 1)!; hence 
the ratio of e x /x m is unbounded by any M. 

8 . Replace n by e" and apply the method of the previous exercise. 

9. If \ f(z)\ < M\z\ m for \z\ < r, then e f(z) < e M '* r = 1 + \z\ m (M + M 2 \z\ m /2\ + 
M 3 |z| 2 m /3! + •••) < 1 + \z\ m (M + M 2 r m /2! + M 3 r 2 m /3! + •••). 

10 . ln(l + 0(z m )) = 0(z m ), if m is a positive integer. Proof: If f(z) = 0(z m ), there 

exist positive numbers r < 1, r' < 1 , and a constant M such that \f(z)\ < M\z\ m < r' 
when \z\ < r. Then |ln(l + f{z))\ < \f{z)\ + ±|/(z )| 2 H < |z| m M(l + \r' -+ ). 

11 . We can apply Eq. ( 12 ) with m = 1 and z = In n/n. This is justified since In n/n < r 
for any given r > 0 , when n is sufficiently large. 

12 . Let/(z) = (ze z /(e z — l)) 1 ^ 2 . If [i/ 2 — fc] were 0(n k ), the stated identity would show 
that [ 2 fc ] f(z) = 0{n k /(k — 1 )!), so f(z) would converge when 2 = 2ni. But f(2ni) — 00 . 

13 . Proof: We may take L — 1/M in the definitions of O and Q. 


SECTION 1.2.11.2 

1. (-Bo + -Bi£ + -B 2 2 2 /2! + ' • • )e z = (Bo + Bi z + B2Z 2 /2\ + ■ • ■ ) + z; apply Eq. 1.2.9-(n). 

2. The function B m+ i({i}) must be continuous, for the integration by parts. 

3. \Rmn\ < \B m /(m)\\ /” \ f ( ' m \x)\dx. [Notes: We have B m (x ) = (-l) m B m (l - x ), 


and Bm(x) is m! times the coefficient of z n 
zt2 /(e z - 1 ) = 1 l<° z/2 


z /(e z — 1). In particular, since 
e*'*/(e* — 1 J = l/(e z/ 2 - 1) — l/(e z — 1) we have Bm(\) — (2 1 ~ m — l)B m . It is not 
difficult to prove that the maximum of \B m -Bm(x) \ for 0 < x < 1 occurs at x = 1 when 
m is even. Now when m = 2k > 4, let us write simply R m and C m for the quantities 
Rmn and C mn . We have R m - 2 = C m + Rm = /"(B m - Bm({x}))f {m) (x) dx/m\, 
and B m - £ m ({ir}) is between 0 and (2 — 2 1_Tn )B m ; hence R m -2 lies between 0 and 
(2 — 2 1-m )C m . It follows that R m lies between — C m and (1 — 2 1 ~ m )C m , a slightly 
stronger result. According to this argument we see that if f l - m+2 \x)f ( ' rn+ ‘ i \x) > 0 
for 1 < x < n, the quantities C m +2 and C m + 4 have opposite signs, while R m has the 
sign of C m + 2 and R m + 2 has the sign of C m + 4 and |R m + 2 | < |C m + 2 |; this proves ( 13 ). 
See J. F. Steffensen, Interpolation (Baltimore: 1937), §14.] 

A V' l.rn n m + 1 ,\^B k m\ m-fc +1 1 n , \ 1 

4 - 2^ k ^ = zrTRBm+iW- 


0< fc <71 

5. It follows that 


(m — k + 1 )! 


; = \Pl lim 


2 .. 2 

k = hm — 


2 2 n (n !) 2 
\fn ( 2 n)! ’ 
n 2 (n — l) 2 . 


.(I ) 2 


m + 1 


2 • 2 • 4 ■ 4 • 


m + 1 


r B „ 


n (n- |) 2 (n 


3)2 
2 / * 


(I ) 2 


= 2tt. 


1 • 3 • 3 • 5 

6 . Assume that c > 0 and consider Ylo<k<n + c )- 

ln(c(c + 1 ) . . . (c + n — 1 )) = (n + c) ln(n + c) — clnc — n — | ln(n + c) + | Inc 


+ 


E 

1 <fc<m 


1 


B k (-l) k 

k(k — 1 ) ^ (n + c) 


1 


+ Rn 
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Also 


B m ({x}) dx 


ln(n-l)! = (n-i)lnn-n + ff + V M_if 

U-v -A 

Now lnr„_i(c) = cln(n — 1) + In (n — 1)! — ln(c ... (c + n — 1)); substituting and letting 
n - > oc. we get 

Bk( — l) k 1 f°° B m {{x}) dx 


In r(c) = — c + (c — 1) In c + a + 


k(k — 1 )c 


r j_ r 

■' £_1 m J 0 


( x + c)' 


1 < fc < m 

This shows that T(c + 1) = ce Inr ^ c ^ has the same expansion we derived for c!. 

7. An n / 2 +"/ 2 +i/i2 e -n 2 /4 where A ig a constant To 0 b ta i n t i lis reS ult, apply Euler’s 
summation formula to J2k=i k In k. A more accurate formula is obtained if we multiply 
the answer above by 

exp(-B 4 /(2 • 3 • 4n 2 ) B 2t /((2t - 2)(2 1 - 1)(2 t)n 2t ~ 2 ) + 0(l/n 2 ')). 

In these formulas, A is the “Kinkelin-Glaisher constant” 1.2824271... [Crelle 57 
(1860), 122-158; Messenger of Math. 7 (1877), 43-47], which can be shown to equal 
e i/i2 C ( i) = (27re 7-c ( 2 )/<(2})i/i2 [ de B ru jj n Asymptotic Methods in Analysis, §3.7], 

8. We have, for example, In (an 2 + bn) = 2 Inn + In a + ln(l + b/(an)). Thus the 
answer to the first question is found to be 2 an 2 lnn + o(lna- l)n 2 +2bnlnn + bn lna + 
Inn + b /(2a) + ^ In a + <r + (3a — 6 2 )b/(6a 2 n) + 0(n 2 ). Massive cancellation occurs 
when we compute the quantity In (cn 2 )! - In (cn 2 - n)! - nine - Inn 2 ! + In (n 2 - n)\ = 
( c — l)/(2c) — (c — l)(2c — l)/(6c 2 n) + 0(n~~). The answer is therefore 


(c 1 ) / (2c) ( l _ (c- l) (2c - 1) 


6c 2 r 


)(l + 0(n- 2 )). 


Incidentally, ( n )/c n ( n ) can be written + aj/(n 2 — j )) where a = 1 — 1 /c. 

9. (a) We have ln(2n)! = (2n + ±) In 2n - 2n + a + ^ + 0(n“ 3 ), and In (n!) 2 = 
(2n + 1) Inn — 2n + 2<j + + 0(n~ 3 ); hence ( 2 ") = exp(2nln2 — ~ hnvn 4* 4. 
0(n“ 3 )) = 2 2 "( 7 rn)- 1 / 2 (l - fn" 1 + ^n- 2 + 0(n/_ 3 )). (b) Since ( 2 n ") = 2 2n { n ~l' 2 ) 

an< ^ ( n ^ ” T(n -(- l/ 2 )/(nT(n)r(l/ 2 )) = n Vi 1 / 2 /-^-, we obtain the same result 

from 1.2.11.1— (16) because 


r 1/2 i = 1 

L1/2J ’ 


1/2 

-1/2J 




1/2 

-3/2 


/1/2 
4 


i 


+2 


3/2 

4 


1 

128' 


Method (b) explains why the denominators in 


i)2n 


n- 2 5n“ 3 

+ 128 + 1024 


21n“ 


399n“ 5 869n~ 

■ + 


32768 262144 4194304 


+ 0(n 


are all powers of 2 [Knuth and Vardi, AMM 97 (1990), 629-630]. 


SECTION 1.2.11.3 

1. Integrate by parts. 

2. Substitute the series for e~ t in the integral. 

3. See Eq. 1.2.9— (11) and exercise 1.2.6-48. 
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4. 1 + \/u is bounded as a function of v. since it goes to zero as v goes from r to 
infinity. Replace it by M and the resulting integral is Me"". 

5. /"( x) = f(x)((n + l/2)(n — \/2)/x 2 — (2 n + l)/x + l) changes sign at the point 
r = n + 1/2 - ^+1/2, so \R\ = 0(f g n \f"{x)\dx) = O(/ 0 r f"(x) dx - f r " f"(x) dx) = 
0(f'(n) - 2/'(r) + /'( 0)) = 0(/(n)/V^). 

6 . It is n n+ B exp ((n + /3)(a/n — a 2 /‘2n 2 + 0(n~ 3 ))), etc. 

7. The integrand as a power series in x~ x has the coefficient of x~ 71 as 0(u 2n ). After 
integration, terms in x~ 3 are Cu 7 /x 3 = 0(x~ 5/l4 ), etc. To get 0(x~ 2 ) in the answer, 
we can discard terms u n /x m with 4m — n > 9. Thus, an expansion of the product 
exp(—u 2 /2x) exp(u 3 /3x 2 ) . . . leads ultimately to the answer 


yx 1 / 4 -^x- 1/4 + ^x- 3/4 +^x- 1 -^x- 5/4 -^x- 3/2 +(^~^)x- 7 ^+0(x- 2 ). 
6 40 12 336 36 V3456 20/ V ’ 


8 . (Solution by Miklos Simonovits.) We have |/(x)| < x if x is large enough. Let 
R(x) = — e ~ h( - u ' x ')'j du be the difference between the two given integrals, 

where g(u,x) = u — x ln(l + u/x) and h(u,x) = u 2 /2x- u 3 /3x 2 H h ( — l) m u m /rra m " 1 . 

Notice that g(u,x) > 0 and h(u,x) > 0 when |w| < x; also g(u, x) = h(u,x) + 
0(u rn+1 /x m ). 

According to the mean value theorem, e“ — e b = (a — b) e c for some c between 
a and b. Therefore |e“ — e 6 | < | a — b\ when a, 6 < 0. It follows that 

/■ 110)1 / rMx r m+ 1 j \ 

-R(aO| < / \g{u,x) - h(u,x)\du = 0[ / — ) 

= 0(x im+2)r - m ) = 0(x~ s ). 


9. We may assume that p / 1, since p = 1 is given by Theorem A. We also assume 
that p ^ 0 , since the case p = 0 is trivial. 

Case 1: p < 1. Substitute t = px( 1 — u) and then v = — ln(l — u) — pu. We have 
du = ((1 — p + pu)/( 1 — u)) du, so the transformation is monotone for 0 < u < 1 , and 
we obtain an integral of the form 



— XV i 

xe dv 


( 


1 — u \ 
1 — p + pu) 


Since the parenthesized quantity is (1 — p) 1 (l — v(l — p) 2 + • • • ), the answer is 


P 

1 -p 


(p, 


1 — p\x 


r(* + i) 



(p — l) 2 x 


+ Q( X - 2 ) 


Case 2: p > 1. This is 1 — f^( ). In the latter integral, substitute t = px( 1 + it), 
then v = pu — ln(l + u ), and proceed as in Case 1. The answer turns out to be the 
same formula as Case 1, plus one. Notice that pe 1-p < 1 , so (pe 1 ~ p ) x is very small. 
The answer to exercise 11 gives another way to solve this problem. 

p~y[ e v _ i 2 i 


10 . 


p - 1 


(p, 


? 1 — p\x 


1 -e~ y - 


x(p — l ) 2 


v r/ ~+o( X - 2 : 


11. First, xQ x (n) + Ri/ X {n) = n! (x/n) n e n ^ x generalizes ( 4 ). Next, we have R x {n) = 
n\ (e x /nx) n 'y(n, nx)/(n — 1)!, generalizing ( 9 ). Since aj(a, x) c* -y(a + 1, x) + e~ x x a we 
can also write R x {n) = 1 + (e x /nx) n 'y(n + 1, nx ), relating this problem to exercise 9. 


506 ANSWERS TO EXERCISES 


1 . 2 . 11.3 


Moreover, we can tackle Q x (n) and R x (n) directly by using Eqs. 1.2.9-(27) and (28) 
to derive series expansions involving Stirling numbers: 

1 + x Q x {n) = ]T x k n l -/n k = £ 

0 k,m 

Rx(n) = '£2 x k n k /(n + 1)* = ^ — 

k,m 

The sums over k are convergent for fixed m when \x\ < 1, and when |x| > 1 we can use 
the relation between Q x (n) and R 1/x {n); this leads to the formulas 


k 

k — m 


l) m f Hml k 
n m \ k J 


Q.r(/l) = 
R x (n ) = 
Q x (n) = 


+ ' • • + 7! + 0 (n - 1 -™) 


1 — x (1 — x) 3 n (1 — x) 2rn+1 n m 

1 x 


1—x (1 — x) 3 n 


(1) T m (x) — 1 — m\ . , 

+ "' + (T- X )2m + l nm +°(" )> lf X ^ 

* + ■ • • + +Q(n- 1 -’ n ), 


1 — x (1 — x) 3 n 
+ ■■■ + 


1 * HTMi) 


R x (n) = + 

n n x n 1-x (1 — x) 3 n (1 - x) 2m + 1 n m 


(1 - x) 2rn+1 n m 

+ 0(n 1 m ), if x > 1. 


Here 


and 


q m (x) 




r m {x) - {{ ™ + // ''' ))x 2 + 


are polynomials whose coefficients are “second-order Eulerian numbers” [CMatfi §6.2: 
see L. Carlitz, Proc. Amer. Math. Soc. 16 (1965), 248-252], The case x =t -1 is 
somewhat delicate, but it can be handled by continuity, because the bound implied by 
0(n~ 1 ~ m ) is independent of x when x < 0. It is interesting to note that - 

Q~i(n) = ( — 1 )"n!/e“n“ tn ( — l) n \f2'irn/e 2n is extremely small. 

12 . 7 (|, \x 2 )/V2. 

13. See P. Flajolet, P. Grabner, P. Kirschenhofer, and H. Prodinger, J. Computational 
and Applied Math. 58 (1995), 103-116. 

15. Expanding the integrand by the binomial theorem, we obtain 1 + Q(n). 

16. Write Q(k) as a sum, and interchange the order of summation using Eq. 1.2.6-(53). 

17. S{n) = vW 2 + | - ^ir/2n - /2n 3 + 0(n~ 2 ). [Note that 

S(n + 1) + P(n) = J2 k > 0 k n ~ k k\/n\, while Q(n) + R(n) = J2k>o n\/kln n ~ k .} 

18. Let S„(x , y ) = Yl k { k ) (x + k) k (y + n — k) n k . Then for n > 0 we have S n (x, y) = 
x T,k (l)( x + k) k -\y + n- k) n ~ k + nJ2 k (VK® + 1 + k) k {y + n - 1 - k) n ~ l ~ k = 
(^ + ^ + n ) n + nS n -i(x + 1, y) by Abel’s formula 1.2.6-(i6); consequently S n (x,y) = 
S k ( x + V + n ) n k - [This formula is due to Cauchy, who proved it using the 
calculus of residues; see his (Euvres (2) 6, 62-73.] The stated sums are therefore equal 
respectively to n n (l + Q{n)) and (n + l)"Q(n +1). 
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19. Suppose C n exists for all n > IV and |/(a:)| < Mx a for 0 < x < r. Let F(x) = 
jx e ~Nt ^ ^hen w j len n > N we have 


pr poo 

\C n \< J e~ nx \f(x)\dx + J e~ < ' n ~ N ^ x e~ Nx f(x) dx 

pr poo 

< M J e~ nx x a dx + (n — N) J e~^ n ~ N ^ x F(x) dx 


< M 


poo p o 

/ e~ nx x a dx + (n — N) sup|F(:r) | / 
JO x>r J r 


~(n — N)x 


dx 


MT(a + l)n 1 “ + sup|F(x)| e ^ n N ' >r = 0(n 1 “). 

x>r 


[E. W. Barnes, Phil. Trans. A206 (1906), 249-297; G. N. Watson, Proc. London Math. 
Soc. 17 (1918), 116-148.] 

20. [C. C. Rousseau, Applied Math. Letters 2 (1989), 159-161.] We have Q(n) + 1 = 
n J 0 °° e~ nx (l + x) n dx = n J 0 °° e ~ n ( x - ln P+ x ')'> dx = n J 0 °° e~ nu g(u) du, by substituting 
u = x — ln(l +x) and letting g(u) — dx/du. Notice that x = Yl'kLi c k(2u) k when u is 
sufficiently small. Hence g(u ) = l 1 kck(2u) k ^ 2_1 + 0(u m ^ 2 ~ 1 ), and we can apply 
Watson’s lemma to Q(n) + 1 — n J 0 °° e~ nu kck(2u) k ^ 2 ~ 1 du. 


SECTION 1.3.1 


1. Four; each byte would then contain 3 4 = 81 different values. 

2. Five, since five bytes is always adequate but four is not. 

3. (0:2); (3:3); (4:4); (5:5). 

4. Presumably index register 4 contains a value greater than or equal to 2000, so that 
a valid memory address results after indexing. 


5. “DIV -80,3(0:5)” or simply “DIV -80,3”. 

6. (a) rA 


- 

JL 

1 

200 

15 


. (b) rI2 < 200. (c) rX 


0 

0 

0 

5 

1 

? 


(d) Undefined; we can’t load such a big value into an index register, (e) rX 


- 

0 

o 

0 

0 

0 


7. Let n = |rAX| be the magnitude of registers A and X before the operation, and 
let d = | V | be the magnitude of the divisor. After the operation the magnitude of rA is 
[n/dj, and the magnitude of rX is n mod d. The sign of rX afterwards is the previous 
sign of rA; the sign of rA afterwards is + if the previous signs of rA and V were the 
same, otherwise it is — . 

Stating this another way: If the signs of rA and V are the same, rA <— [rAX/Vj 
and rX rAX mod V. Otherwise rA 4- [rAX/V] and rX 4— rAX mod —V. 


+ 1 

0 

617 

0 

1 

; rX 4- 

- 

0 

0 

0 

1 

0 


9. ADD, SUB, DIV, NUM, J0V, JN0V, INCA, DECA, INCX, DECX. 

10. CMPA, CMP1, CMP2, CMP3, CMP4, CMP5, CMP6, CMPX. (Also FCMP, for floating point.) 

11. MOVE, LD1, LD1N, INC1, DEC1, ENT1, ENN1. 

12. INC3 0,3. 
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13. “JOV 1000’' makes no difference except time. “JNOV 1001” makes a different 
setting of rJ in most cases. “JN0V 1000” makes an extraordinary difference, since 
it may lock the computer in an infinite loop. 

14. NOP with anything; ADD, SUB with F = (0:0) or with address equal to * (the 
location of the instruction) and F = (3:3); HLT (depending on how you interpret the 
statement of the exercise); any shift with address and index zero; SLC or SRC with 
index 0 and address a multiple of 10; MOVE with F = 0; STJ *(0:0), STZ *(0:0). and 
STZ *(3:3); JSJ *+l; any of the INC or DEC instructions with address and index zero. 
But “ENT1 0,1” is not always a no-op, because it might change rll from -0 to +0. 

15. 70; 80; 120. (The block size times 5.) 

16. (a) STZ 0; ENT1 1; MOVE 0(49); MOVE 0(50). If the byte size were known to equal 
100, only one MOVE instruction would have been necessary, but we are not allowed to 
make assumptions about the byte size, (b) Use 100 STZ’s. 

17. (a) STZ 0,2; DEC2 1; J2NN 3000. 

(b) STZ 0 
ENT1 1 
JMP 3004 
(3003) MOVE 0(63) 

(3004) DEC2 63 

J2P 3003 

INC2 63 

ST2 3008(4:4) 

(3008) MOVE 0 | 


(A slightly faster, but quite preposterous, program uses 993 STZ’s: JMP 3995: STZ 1,2: 
STZ 2,2; ...; STZ 993,2; J2N 3999; DEC2 993; J2NN 3001; ENN1 0,2; JMP 3000,1.) 

18. (If you have correctly followed the instructions, an overflow will occur on the ADD. 

irison is 
, rll is 


set EQUAL, rA is set to 

- 

30 

30 

30 

30 

30 

, rX is set to 

- 


30 

30 

30 

30 


begins in location 0000.) 


19. 42u =(2 + 1 + 2 + 2 + 1 + 1 + 1 + 2 + 2 + 1 + 2 + 2 + 3+10+ 10)u. 

20. (Solution by H. Fukuoka.) 

(3991) ENT1 0 

MOVE 3995 (standard F for MOVE is 1) 

(3993) MOVE 0(43) (3999 = 93 times 43) 

JMP 3993 
(3995) HLT 0 | 


21. (a) Not unless it can be set to zero by external means (see the “GO button”, 
exercise 26), since a program can set rJ t— N only by jumping from location N — 1. 
(b) LDA -1,4 
LDX 3004 
STX -1,4 
JMP -1,4 
(3004) JMP 3005 
(3005) STA -1,4 | 
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22. Minimum, time: If b is the byte size, the assumption that |X 13 | < b 5 implies that 
A' 2 < b , so X 2 can be contained in one byte. The following ingenious solution due to 
Y. N. Patt makes use of this fact. The sign of rA is the sign of X . 


(3000) 


(3500) 

(3501) 


LDA 2000 

MUL 2000(1:5) r A rX 

STX 3500(1:1) ~ 


SRC 

1 

X 2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

MUL 

3500 

A 4 

0 

0 

0 

0 

0 

0 

0 

0 

STA 

3501 

A 4 

0 

0 

0 

0 

0 

0 

0 

0 

ADD 

2000 

A 4 

0 

0 

A 

0 

0 

0 

0 

0 

MUL 

3501(1:5) 

A 8 

0 

A 5 

0 

0 

0 

STX 

3501 

A 8 

0 

A 5 

0 

0 

0 

MUL 

3501(1:5) 

0 

A 13 

0 

0 

0 

0 

SLAX 

1 

A 13 

0 

0 

0 

0 

0 


HLT 0 
NOP 0 
NOP 0 | 


space = 14; time = 54 u, not counting the HLT. 


At least five multiplications are “necessary,” according to the theory developed in 
Section 4.6.3, yet this program uses only four! And in fact there is an even better 
solution below. 


DEC4 1 
J4P 3002 
HLT 0 | 

space = 7; time = 171u. 

True minimum time: As R. W. Floyd points out, the conditions imply that |X| < 5, 
so the minimum execution time is achieved by referring to a table: 

(3000) LD1 2000 

LDA 3500,1 
HLT 0 
(3495) ( — 5) 13 
(3496) (— 4) 13 


Minimum space: (3000) ENT4 12 

LDA 2000 
(3002) MUL 2000 
SLAX 5 


(3505) (+5) 13 | 

space = 14; time = 4 u. 


23. The following solution by R. D. Dixon appears to satisfy all the conditions: 


(3000) ENT1 4 
(3001) LDA 200 
SRA 0,1 
SRAX 1 


+ 

1 

0 

0 

0 

0 


DEC1 1 
JINN 3001 
SLAX 5 
HLT 0 | 


24. (a) DIV 3500, where 3500 
(b) SRC 4; SRA 1; SLC 5. 
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25. Some ideas: (a) Obvious things like faster memory, more input-output devices, (b) 
The I field could be used for J-register indexing, and/or multiple indexing (to specify 
two different index registers) and/or “indirect addressing’’ (exercises 2. 2. 2-3, 4, 5). 
(c) Index registers and J register could be extended to a full five bytes; therefore 
locations with higher addresses could be referred to only by indexing, but that would 
not be so intolerable if multiple indexing were available as in (b). (d) An interrupt 
capability could be added, using negative memory addresses as in exercise 1.4.4 18. 
(e) A real time clock could be added, in a negative memory address, (f) Bitwise 
operations, jumps on register even or odd, and binary shifts could be added to binary 
versions of MIX (see, for example, exercises 2.5-28, 5.2.2-12, and 6.3-9; also Program 
4.5.2B, 6.4-(24), and Section 7.1). (g) An “execute” command, meaning to perform 
the instruction at location M, could be another variant of C = 5. (h) Another variant 
of C = 48, . . . , 55 could set Cl 4— register : M. 


26. It is tempting to use a (2:5) field to get at columns 7-10 of the card, but this 
cannot be done since 2-8 + 5 = 21. To make the program easier to follow, it is 
presented here in symbolic language, anticipating Section 1.3.2. 



BUFF 

EQU 

29 



0RIG 

0 

00 

L0C 

IN 

16(16) 

01 

READ 

IN 

BUFF (16) 

02 


LD1 

0(0:0) 

03 


JBUS 

*(16) 

04 


LDA 

BUFF+1 

05 

= 1= 

SLA 

1 

06 


SRAX 

6 

07 

=30= 

NUM 

30 

08 


STA 

L0C 

09 


LDA 

BUFF+1 (1:1) 

10 


SUB 

=30=(0:2) 

11 

LOOP 

LD3 

L0C 

12 


JAZ 

0,3 

13 


STA 

BUFF 

14 


LDA 

L0C 

15 


ADD 

=1= (0 : 2) 

16 


STA 

L0C 

17 


LDA 

BUFF+3 ,1(5:5! 

18 


SUB 

=25=(0:2) 

19 


STA 

0, 3(0:0) 

20 


LDA 

BUFF+2 , 1 

21 


LDX 

BUFF+3 , 1 

22 

=25= 

NUM 

25 

23 


STA 

0, 3(1:5) 

24 


MOVE 

0,1(2) 

25 


LDA 

BUFF 

26 


SUB 

=1= (0 : 2) 

27 


JAP 

LOOP 

28 


JMP 

READ 


Buffer area is 0029-0044 

Read in second card. 
Read next card, 
rll +- 0. 

Wait for read to finish. 
rA 4 — columns 6-10. 

rAX 4— columns 7-10. 

L0C 4— starting location. 

i'I3 4 - L0C. 

Jump, if transfer card. 
BUFF +- count. 


L0C 4 — L0C + 1. 

Store the sign. 


Store the magnitude, 
rll 4 - rll + 2. (!) 

Decrease the count. 

Repeat until the count is zero. 
Now read a new card. 


characters 
punched on card: 

LjO u 06 

ljZlj06 

UUULjI 

uC u 04 

u0 u EH 

uAuuE 

U F U CF 

uO uu E 

uuljEU 

u0 u IH 
U G U BB 
uuuEJ 
uuCA • 

U Z U EU 

uuuEH 

U E U BA 


uuuEU 

U 2A-H 

uS u BB 

uljCuU 

U 1AEH 

U 2AEN 

uVuljE 

uuCLU 

uljABG 

uZ u EH 

U E U BB 

u JljB . 

uAuu9 I 
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SECTION 1.3.2 

1. ENTX 1000; STX X. 

2. The STJ instruction in line 03 resets this address. (It is conventional to denote the 
address of such instructions by “*” , both because it is simple to write, and because it 
provides a recognizable test of an error condition in a program, in case a subroutine 
has not been entered properly because of some oversight. Some people prefer 

3. Read in 100 words from tape unit zero; exchange their maximum with the last of 
them; exchange the maximum of the remaining 99 with the last of those; etc. Eventually 
the 100 words will become completely sorted into nondecreasing order. The result is 
then written onto tape unit one. (Compare with Algorithm 5.2.3S.) 


4. Nonzero locations: 


3000: 

+ 

0000 

00 

18 

35 

3001: 

+ 

2051 

00 

05 

09 

3002: 

+ 

2050 

00 

05 

10 

3003: 

+ 

0001 

00 

00 

49 

3004: 

+ 

0499 

01 

05 

26 

3005: 

+ 

3016 

00 

01 

41 

3006: 

+ 

0002 

00 

00 

50 

3007: 

+ 

0002 

00 

02 

51 

3008: 

+ 

0000 

00 

02 

48 

3009: 

+ 

0000 

02 

02 

55 

3010: 

- 

0001 

03 

05 

04 

3011: 

+ 

3006 

00 

01 

47 

3012: 

- 

0001 

03 

05 

56 

3013: 

+ 

0001 

00 

00 

51 

3014: 

+ 

3008 

00 

06 

39 

3015: 

+ 

3003 

00 

00 

39 

3016: 

+ 

1995 

00 

18 

37 

3017: 

+ 

2035 

00 

02 

52 

3018: 

- 

0050 

00 

02 

53 

3019: 

+ 

0501 

00 

00 

53 

3020: 

- 

0001 

05 

05 

08 


3021: 

+ 

0000 

00 

01 

05 

3022: 

+ 

0000 

04 

12 

31 

3023: 

+ 

0001 

00 

01 

52 

3024: 

+ 

0050 

00 

01 

53 

3025: 

+ 

3020 

00 

02 

45 

3026: 

+ 

0000 

04 

18 

37 

3027: 

+ 

0024 

04 

05 

12 

3028: 

+ 

3019 

00 

00 

45 

3029: 

+ 

0000 

00 

02 

05 

0000: 

+ 





2 

1995: 

+ 

06 

09 

19 

22 

23 

1996: 

+ 

00 

06 

09 

25 

05 

1997: 

+ 

00 

08 

24 

15 

04 

1998: 

+ 

19 

05 

04 

00 

17 

1999: 

+ 

19 

09 

14 

05 

22 

2024: 

+ 




2 

035 

2049: 

+ 

2010 

2050: 

+ 

3 

2051: 

- 

499 


(the latter two may be interchanged, with 
corresponding changes to 3001 and 3002) 


5. Each OUT waits for the previous printer operation to finish (from the other buffer). 

6. (a) If n is not prime, by definition n has a divisor d with 1 < d < n. If d > \/n, 

then n/d is a divisor with 1 < n/d < \J7i. (b) If N is not prime, N has a prime 

divisor d with 1 < d < \/N. The algorithm has verified that N has no prime divisors < 
p = PRIMEfK] ; also N = pQ + R<pQ + p<p 2 +p< (p + l) 2 . Any prime divisor of N is 
therefore greater than p + 1 > \/N. 
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We must also prove that there will be a sufficiently large prime less than N when N 
is prime, namely that the (k + l)st prime p k+1 is less than p\ + p k - otherwise K would 
exceed J and PRIME [K] would be zero when we needed it to be large. The necessary 
proof follows from Bertrand s postulate ’ : If p is prime there is a larger prime less 
than 2 p. 

7 . (a) It refers to the location of line 29. (b) The program would then fail; line 14 
would refer to line 15 instead of line 25; line 24 would refer to line 15 instead of line 12. 

8. It prints 100 lines. If the 12000 characters on these lines were arranged end to 
end, they would reach quite far and would consist of five blanks followed by five A’s 
followed by ten blanks followed by five A’s followed by fifteen blanks . . . followed by 
5k blanks followed by five A’s followed by 5 {k + 1) blanks . . . until 12000 characters 
have been printed. The third-from-last line ends with AAAAA and 35 blanks; the final 
two lines are entirely blank. The total effect is one of OP art. 

9 . The (4:4) field of each entry in the following table holds the maximum F setting; 
the (1:2) field is the location of an appropriate validity-check routine. 


B 

EQU 

1(4:4) 

BEGIN 

LDA 

INST 


BMAX 

EQU 

B-l 


CMPA 

VALID(3 : 3) 


UMAX 

EQU 

20 


JG 

BAD 

I field > 6? 

TABLE 

NOP 

GOOD (BMAX) 


LD1 

INST (5 : 5) 



ADD 

FL0AT(5:5) 


DEC1 

64 



SUB 

FLOAT (5 : 5) 


JINN 

BAD 

C field > 64? 


MUL 

FL0AT(5 : 5) 


CMPA 

TABLE+64, 1(4:4) 



DIV 

FL0AT(5:5) 


JG 

BAD 

F field > F max? 


HLT 

GOOD 


LD1 

TABLE+64, 1(1: 2) 

Jump to special 


SRC 

GOOD 


JMP 

0,1 

routine. 


MOVE 

MEMORY (BMAX) 

FLOAT 

CMPA 

VALID (4: 4) 

F = 6 allowed on 


LDA 

FIELD (5: 5) 


JE 

MEMORY 

arithmetic op 




FIELD 

ENTA 

0 



STZ 

FIELD(5 : 5) 


LDX 

INST (4: 4) 

This is a tricky 


JBUS 

MEMORY (UMAX) 


DIV 

=9= 

way to check 


IOC 

GOOD (UMAX) 


STX 

*+1(0:2) 

for a valid 


IN 

MEMORY (UMAX) 


INCA 

0 

partial field. 


OUT 

MEMORY (UMAX) 


CMPA 

=5= 



JRED 

MEMORY (UMAX) 


JG 

BAD 



JLE 

MEMORY 

MEMORY 

LDX 

INST(3:3) 



JANP 

MEMORY 


JXNZ 

GOOD 

If I = 0, 





LDX 

INST(0:2) 

ensure the 


JXNP 

MEMORY 


JXN 

BAD 

address is a 


ENNA 

GOOD 


CMPX 

=3999= 

valid memory 





JLE 

GOOD 

location. 


ENNX 

GOOD 


JMP 

BAD 



CMPA 

FL0AT(5:5) 

VALID 

CMPX 

3999,6(6) 

1 


CMP1 

FIELD (5: 5) 






CMPX 

FIELD (5: 5) 






10 . The catch to this problem is that there may be several places in a row or column 
where the minimum or maximum occurs, and each is a potential saddle point. 
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Solution 1: In this solution we run through each row in turn, making a list of all columns 
in which the row minimum occurs and then checking each column on the list to see if 
the row minimum is also a column maximum. rX = current min; rll traces through the 
matrix, going from 72 down to zero unless a saddle point is found; rI2 = column index 
of rll; rI3 = size of list of minima. Notice that in all cases the terminating condition 
for a loop is that an index register is < 0. 


* SOLUTION : 

L 


A10 

EQU 

1008 

Location of a.io 

LIST 

EQU 

1000 


START 

ENT1 

9*8 

Begin at the lower right corner. 

ROWMIN 

ENT2 

8 

Now rll is at column 8 of its row. 

2H 

LDX 

A10, 1 

Candidate for row minimum 


ENT3 

0 

List empty 

4H 

INC3 

1 



ST2 

LIST.3 

Put column index in list. 

1H 

DEC1 

1 

Go left one. 


DEC2 

1 



J2Z 

COLMAX 

Done with row? 

3H 

CMPX 

A10 , 1 



JL 

IB 

Is rX still minimum? 


JG 

2B 

New minimum? 


JMP 

4B 

Remember another minimum. 

COLMAX 

LD2 

LIST, 3 

Get column from list. 


INC2 

9*8-8 


1H 

CMPX 

A10 , 2 



JL 

NO 

Is row min < column element? 


DEC2 

8 



J2P 

IB 

Done with column? 

YES 

INC1 

A10+8,2 

Yes; rll address of saddle. 


HLT 



NO 

DEC3 

1 

Is list empty? 


J3P 

COLMAX 

No; try again. 


J1P 

ROWMIN 

Have all rows been tried? 


HLT 


Yes; rll = 0, no saddle. | 


Solution 2: An infusion of mathematics gives a different algorithm. 

Theorem. Let R(i) = ruin, a^, C(j) = max, a lJ . The element cn 0 j 0 is a saddle point 
if and only if R(i 0 ) = max; R(i) = C(j 0 ) = min., C(j). 

Proof. If ai 0 j 0 is a saddle point, then for any fixed i, R(io) = C(jo) > aij 0 > R(i)', so 
R(i o) = maxi R(i). Similarly C(jo) = mim, C(j). Conversely, we have R(i) < <Hj < 
C(j) for all i and j; hence R{io) = C(jo) implies that aj 0J0 is a saddle point. | 

(This proof shows that we always have maxi R(i) < min jC(j). So there is no 
saddle point if and only if all the R' s are less than all the C’s.) 

According to the theorem, it suffices to find the smallest column maximum, then 
to search for an equal row minimum. During Phase 1, rll = column index; rI2 runs 
through the matrix. During Phase 2, lil s possible answer; rI2 runs through the 
matrix; rI3 = row index times 8; rI4 = column index. 
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* SOLUTION 2 


CMAX 

EQU 1000 


A10 

EQU CMAX+8 


PHASE 1 

ENT1 8 

Start at column 8. 

3H 

ENT2 9*8-8 ,1 
JMP 2F 


1H 

CMPX A10,2 

JGE *+2 

Is rX still maximum? 

2H 

LDX A10,2 
DEC2 8 

J2P IB 

New maximum in column 


STX CMAX+8 , 2 

Store column maximum. 


J2Z IF 

First time? 


CMPA CMAX+8 ,2 
JLE *+2 

rA still min max? 

1H 

LDA CMAX+8 ,2 



DEC1 1 

J1P 3B 

Move left a column. 

PHASE2 

ENT3 9*8-8 

At this point rA = min^ C(j) 

3H 

ENT2 8,3 

ENT4 8 

Prepare to search a row. 

1H 

CMPA A10,2 

Is min,' C{j) > a[i,j ]? 


JG NO 

JL 2F 

No saddle in this row 


CMPA CMAX, 4 

JNE 2F 

Is a[i,j] = C(j)? 


ENT1 A10.2 

Remember a possible saddle point 

2H 

DEC4 1 

DEC2 1 

J4P IB 

Move left in row. 


HLT 

A saddle point was found. 

NO 

DEC3 8 



J3P 3B 

ENT1 0 

Try another row. 


HLT 

rll = 0; no saddle. | 


We leave it to the reader to invent a still better solution in which Phase 1 records 
all possible rows that are candidates for the row search in Phase 2. It is not necessary 
to search all rows, just those i 0 for which C(j 0 ) = min., C{j) implies a iojo = C(j 0 ). 
Usually there is at most one such row. 

In some trial runs with elements selected at random from {0, 1,2, 3, 4}, solution 1 
required approximately 730m to run, while solution 2 took about 530m. Given a matrix 
of all zeros, solution 1 found a saddle point in 137m, solution 2 in 524m. 

If an m x n matrix has distinct elements, and m > n. we can solve the problem 
by looking at only 0(m + n) of them and doing 0(m log n) auxiliary operations. See 
Bienstock, Chung, Fredman, Schaffer, Shor, and Suri, AMM 98 (1991), 418-419. 

11 . Assume an m x n matrix, (a) By the theorem in the answer to exercise 10. all 
saddle points of a matrix have the same value, so (under our assumption of distinct 
elements) there is at most one saddle point. By symmetry the desired probability is 
mn times the probability that a u is a saddle point. This latter is l/(mn)! times the 
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number of permutations with a 12 > a n , . . . , a ln > an, an > a 2 i, . . . , an > a mi ; this 
is l/(ra + n — 1)! times the number of permutations of m + n — 1 elements in which 
the first is greater than the next (m — 1) and less than the remaining (n — 1), namely 
(m — 1)! ( n — 1)!. The answer is therefore 

mn(m — 1)! (?r — l)!/(m + n — 1)! = (m + n ) j " l . 

In our case this is 17/ (g 7 ), only one chance in 1430. (b) Under the second assumption, 
an entirely different method must be used. The probability equals the probability that 
there is a saddle point with value zero plus the probability that there is a saddle point 
with value one. The former is the probability that there is at least one column of 
zeros; the latter is the probability that there is at least one row of ones. The answer 
is (1 - (1 - 2~ m ) n ) + (1 - (1 - 2~ n ) m )- in our case it comes to 924744796234036231/ 
18446744073709551616, about 1 in 19.9. An approximate answer is n2~ m + m2 - ". 

12. M. Hofri and P. Jacquet [Algorithmica 22 (1998), 516-528] have analyzed the case 
when the m x n matrix entries are distinct and in random order. The running times of 
the two MIX programs are then respectively (6mn+5mif rl +8m+6 + 5(m+l)/(n-l))u+ 
0((m + n) 2 / ( m ^ n ) ) and (5 mn + 2nH m + 7m + 7 n + 9 H n )u + 0(1 /n) + 0((logn) 2 /ra), 
as m — > oo and n — > oo, assuming that (log n)/m — > 0. 

13. * CRYPTANALYST PROBLEM (CLASSIFIED) 


TAPE 

EQU 

20 

Input unit number 

TYPE 

EQU 

19 

Output unit number 

SIZE 

EQU 

14 

Input block size 

0SIZE 

EQU 

14 

Output block size 

TABLE 

EQU 

1000 

Table of counts 


0RIG 

TABLE 

(initially zero 


CON 

-1 

except entries for 


0RIG 

TABLE+46 

blank space and 


CON 

-1 

asterisk) 


0RIG 

2000 


BUF1 

0RIG 

*+SIZE 

First buffer area 


CON 

-1 

“Sentinel” at end of buffer 


CON 

*+l 

Reference to second buffer 

BUF2 

0RIG 

*+SIZE 

Second buffer 


CON 

-1 

“Sentinel” 


CON 

BUF1 

Reference to first buffer 

BEGIN 

IN 

BUF1 (TAPE) 

Input first block. 


ENT6 

BUF2 


1H 

IN 

0,6 (TAPE) 

Input, next block. 


LD6 

SIZE+1,6 

During this input, prepare 


ENT 5 

0,6 

to process the previous one. 


JMP 

4F 



2H INCA 1 

STA TABLE, 1 Update table entry. 

3H SLAX 1 

STA *+1(2:2) rll t— next char. 

ENT1 0 

LDA TABLE , 1 

JANN 2B Normal character? 


main 
loop, 
should 
run as 
fast as 
possible 
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J1NZ 

3F 

Asterisk? 


JXP 

3B 

Skip over a blank. 


INC5 

1 


4H 

LDX 

0,5 

rX <— five chars. 


JXNN 

3B 

Jump if not a sentinel. 


JMP 

IB 

Done with block. 

3H 

ENT1 

1 

Begin the endgame: rll +- “A”. 

2H 

LDA 

TABLE, 1 



JANP 

IF 

Skip zero answers. 


CHAR 


Convert to decimal. 


JBUS 

* (TYPE) 

Wait till the typewriter is ready. 


ST1 

CHAR (1:1) 



STA 

CHAR(4 : 5) 



STX 

FREQ 



OUT 

ANS (TYPE) 

Type one answer. 

1H 

CMP1 

=63= 



INC1 

1 

Up to 63 character 


JL 

2B 

codes are counted 


HLT 



ANS 

ALF 


The output buffer 


ALF 



CHAR 

ALF 

C NN 


FREQ 

ALF 

NNNNN 



ORIG 

ANS+OSIZE 

Rest of buffer is blank 


END 

BEGIN 

The literal constant =63= comes 


For this problem, buffering of output is not desirable since it could save at most 7 u of 
time per line output. 

14. To make the problem more challenging, the following solution due in part to 
J. Petolino uses a lot of trickery in order to reduce execution time. Can the reader 
squeeze out any more microseconds? 


* DATE OF EASTER 
EASTER 


CPLUS60 


STJ 

EASTX 


STX 

Y 


ENTA 

0 

El. 

DIV 

= 19= 


STX 

GMINUS1 (0 : 2) 


LDA 

Y 

E2. 

MUL 

=1/7100+1= 

(see 

INCA 

61 

below) 

STA 

CPLUS60C1 : 2) 


MUL 

=3//4+l= 


STA 

XPLUS57C1 : 2) 


ENTA 

* 


MUL 

=8//25+l= 

rA <- Z + 24. 

ENT2 

* 

E5. 

ENT1 

1,2 

rll +- G. 

INC2 

1,1 


INC2 

0,2 



GMINUS1 
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INC2 

0,1 



INC2 

0,2 



INC2 

773,1 

rI2 +- 11G + 773. 

XPLUS57 

INCA 

, 2 

rA <- 11G + Z - X + 20 + 24 • 30 (> 0). 


SRAX 

5 



DIV 

=30= 

rX +- E. 


DECX 

24 



JXN 

4F 



DECX 

1 



JXP 

2F 



JXN 

3F 



DEC1 

11 



J1NP 

2F 


3H 

INCX 

1 


2H 

DECX 

29 

E6. 

4H 

STX 

20MINUSN (0 : 2) 



LDA 

Y 

E4. 


MUL 

=1/ / 4+1= 



ADD 

Y 



SUB 

XPLUS57 (1:2) 

rA<—D — 47. 

20MINUSN 

ENN1 

* 



INCA 

67,1 

E7. 


SRAX 

S 

T X<r- D + N 


DIV 

=7= 



SLAX 

5 



DECA 

-4,1 

rA +- 31 — N 


JAN 

IF 

E8. 


DECA 

31 



CHAR 




LDA 

MARCH 

I 


JMP 

2F 


1H 

CHAR 




LDA 

APRIL 


2H 

JBUS 

*(18) 



STA 

MONTH 



STX 

DAY(1 : 2) 



LDA 

Y 



CHAR 




STX 

YEAR 



OUT 

ANS (18) 

Print 

EASTX 

JMP 

* 


MARCH 

ALF 

MARCH 


APRIL 

ALF 

APRIL 


ANS 

ALF 


I 1 

DAY 

ALF 

DD 


MONTH 

ALF 

MMMMM 



ALF 

> 

| 

YEAR 

ALF 

YYYYY 



ORIG 

*+20 



518 ANSWERS TO EXERCISES 


1.3.2 


BEGIN ENTX 1950 “driver” 

ENT6 1950-2000 routine, 

JMP EASTER uses the 

INC6 1 subroutine 

ENTX 2000,6 above. 

J6NP EASTER+1 
HLT 

END BEGIN | 


A rigoious justification for the change from division to multiplication in several places 
can be based on the fact that the number in rA is not too large. The program works 
with all byte sizes. 

[To calculate Easter in years < 1582, see CACM 5 (1962), 209-210. The first 
systematic algorithm for calculating the date of Easter was the canon paschalis due to 
Victorius of Aquitania (a.D. 457). There are many indications that the sole nontrivial 
application of arithmetic in Europe during the Middle Ages was the calculation of 
Easter date, hence such algorithms are historically significant. See Puzzles and Para- 
doxes by T. H. O’Beirne (London: Oxford University Press, 1965), Chapter 10, for 
further commentary; and see the book Calendrical Calculations by E. M. Reingold and 
N. Dershowitz (Cambridge Univ. Press, 2001) for date-oriented algorithms of all kinds.] 

15. The first such year is A.D. 10317, although the error almost leads to failure in 
A.D. 10108 + 191- for 0 < k < 10. 

Incidentally, T. H. O’Beirne pointed out that the date of Easter repeats with 
a period of exactly 5,700,000 years. Calculations by Robert Hill show that the most 
common date is April 19 (220400 times per period), while the earliest and least common 
is March 22 (27550 times); the latest, and next-to-least common, is April 25 (42000 
times). Hill found a nice explanation for the curious fact that the number of times any 
particular day occurs in the period is always a multiple of 25. 

16. Work with scaled numbers, R n = 10 n r n . Then R n {\/m) = R if and only if 
10 n /(R + |) < m < 10 n /(R - |); thus we find m h = [2 ■ 10 n /(2 R - 1)J. 

* SUM OF HARMONIC SERIES 


BUF 

0RIG 

*+24 


START 

ENT2 

0 



ENT1 

3 

5 — n 


ENTA 

20 


OUTER 

MUL 

= 10= 



STX 

CONST 

2 • 10 n 


DIV 

=2= 



ENTX 

2 



JMP 

IF 


INNER 

STA 

R 



ADD 

R 



DECA 

1 



STA 

TEMP 

■2R - 1 


LDX 

CONST 



ENTA 

0 



DIV 

TEMP 



INCA 

1 



STA 

TEMP 

mh + 1 
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SUB 

M 


MUL 

R 


SLAX 

5 


ADD 

S 


LDX 

TEMP 


1H STA 

S 

Partial sum 

STX 

M 

m = m e 

LDA 

M 


ADD 

M 


STA 

TEMP 


LDA 

CONST 


ADD 

M 

Compute R = R n (l/m) = 

SRAX 

5 

[(2 • 10" + m) / (2m)J . 

DIV 

TEMP 


JAP 

INNER 

R > 0? 

LDA 

S 

10 n S n 

CHAR 



SLAX 

0,1 

Neat formatting 

SLA 

1 


INCA 

40 

Decimal point 

STA 

BUF,2 


STX 

BUF+1,2 


INC2 

3 


DEC1 

1 


LDA 

CONST 


JINN 

OUTER 


OUT 

BUF(18) 


HLT 



END 

START 

1 

The output is 



0006.16 

0008.449 0010.7509 0013.05363 

in 65595u plus output time. (It would be faster to calculate R n (l/m) directly when 

m < 10 n ' 2 \/2, and then to apply the suggested procedure.) 

17. Let N = |_2-10 n /(2m 

+ 1)J. Then 5 n = H N +O(N/l0 n ) + '^2™ =1 ( [2T0’ l /(2fc — 1)J — 

[2 • 10"/(2fc + l)J)fc/10 n 

— Hj\j -f- 0 ( 777 . + O(777/10 n ) — 1 + 2H 2 m — H m = n In 10 H- 

27 — 1 + 2 In 2 + 0(10 n/ ) if we sum by parts and set m « 10 n ' 2 . 

Incidentally, the next several values are 56 = 15.356262, S7 = 17.6588276, 5s = 

19.96140690, Sg 

= 22.263991779, and Sio = 24.5665766353; our approximation to Sio 

is « 24.566576621, which 

is closer than predicted. 

18. FAREY STJ 

9F 

Assume that rll contains n, where n > 1. 

STZ 

X 

x 0 <- 0 . 

ENTX 

1 


STX 

Y 

Vo <r- 1. 

STX 

X+l 

XI 4— 1. 

ST1 

Y+l 

yi 4- n. 

ENT 2 

0 

k 4- 0. 

I 
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LDX 

Y,2 


INCX 

0,1 


ENTA 

0 


DIV 

Y+1,2 


STA 

TEMP 

L (Vk + n)/y k+ ij 

MUL 

Y+1,2 


SLAX 

5 


SUB 

Y,2 


STA 

Y+2,2 

Vk+ 2 

LDA 

TEMP 


MUL 

X+1,2 


SLAX 

5 


SUB 

X,2 


STA 

X+2,2 

X k +2 

CMPA 

Y+2,2 

Test if x k+2 < y k + 2 - 

INC2 

1 

k +- k + 1. 

JL 

IB 

If so, continue. 

JMP 

* 

Exit from subroutine 


19. (a) Induction, (b) Let k > 0 and X — ax k + x — x k , Y = ay k +\ — yk • where 
a = L (j/fc + 77 ')/ 2/fc+iJ • By part (a) and the fact that 0 < Y < n, we have A' ± Y and 
X/Y > x fc+ i/y fc+ i. So if X/Y ^ x k+2 /yk+ 2 we have, by definition, A"/Y > x k+2 /y k+ 2 . 
But this implies that 


1 

__ Xy k+ 1 - Yx k+ 1 

_ X 

+ 1 

Yy k +i 

Yy k + 1 

~ Y 

Vk+1 


( X Xk+2 \ 

f X k+2 

Xk+1 \ 


\Y yk+2j + 

\Vk+ 2 

Vk+1 J 


> 1 + 1 _ yk± i + y > n > 1_ 

~ 5 'y *:+2 2 /fc+iJ/A :+2 Yy k+x y k+2 Yy k+1 y k+ 2 ~ Yy k + 1 ’ 

Historical notes C. Haros gave a (more complicated) rule for constructing such 
sequences, in J. de 1’EcoIe Polytechnique 4, 11 (1802), 364-368; his method was correct, 
but his proof was inadequate. Several years later, the geologist John Farey indepen- 
dently conjectured that is always equal to (aj*—! + x k +i)/(y k -i + y k + i) [Philos. 

Magazine and Journal 47 (1816), 385-386]; a proof was supplied shortly afterwards by 
A. Cauchy [Bull. Societe Philomathique de Paris (3) 3 (1816), 133-135], who attached 
Farey’s name to the series. For more of its interesting properties, see G. H. Hardy and 
E. M. Wright, An Introduction to the Theory of Numbers, Chapter 3. 

20. * TRAFFIC SIGNAL PROBLEM 


BSIZE 

EQU 

1(4:4) 

Bytesize 

2BSIZE 

EQU 

2(4:4) 

Twice bytesize 

DELAY 

STJ 

IF 

If rA contains n, 


DECA 

6 

this subroutine 


DECA 

2 

waits max(n, 7 )u 


JAP 

*-l 

exactly, not including 


JAN 

NOP 

*+2 

the jump to the subroutine 

1H 

JMP 

* 
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FLASH STJ 

2F 

4 

This subroutine flashes the 

ENT 2 

8 

5 

appropriate DON ’ T WALK light 

1H LDA 

=49991= 

7 


JMP 

DELAY 

8 


DECX 

0,1 

9 

Turn light off. 

LDA 

=49996= 

2 


JMP 

DELAY 

3 


INCX 

0,1 

4 

“DON’T WALK” 

DEC2 

1 

1 


J2Z 

IF 

2 

Repeat eight times. 

LDA 

=49993= 

4 


JMP 

IB 

5 

Get back in synch. 

1H LDA 

=399992= 

4 

Set amber 2u after exit. 

JMP 

DELAY 

5 


2H JMP 

* 

6 


WAIT JNOV 

* 

5 

Del Mar green until tripped 

TRIP INCX 

BSIZE 

6 

DON’T WALK on Del Mar 

ENT1 

2BSIZE 

1 


JMP 

FLASH 

2 

Flash Del Mar. 

LDX 

BAMBER 

8 

Amber on boulevard 

LDA 

=799995= 

2 


JMP 

DELAY 

3 

Wait 8 seconds. 

LDX 

AGREEN 

5 

Green for avenue 

LDA 

=799996= 

2 


JMP 

DELAY 

3 

Wait 8 seconds. 

INCX 

1 

4 

DON’T WALK on Berkeley 

ENT1 

2 

1 


JMP 

FLASH 

2 

Flash Berkeley. 

LDX 

AAMBER 

8 

Amber on avenue 

JOV 

*+l 

1 

Cancel redundant trip. 

LDA 

=499994= 

3 


JMP 

DELAY 

4 

Wait 5 seconds. 

BEGIN LDX 

BGREEN 

6 

Green on boulevard 

LDA 

=1799994= 

2 


JMP 

DELAY 

3 

Wait at least 18 

JMP 

WAIT 

4 

seconds. 

AGREEN ALF 

CABA 


Green for avenue 

AAMBER ALF 

CBBB 


Amber for avenue 

BGREEN ALF 

ACAB 


Green for boulevard 

BAMBER ALF 

BCBB 


Amber for boulevard 

END 

BEGIN 


1 

22. * JDSEPHUS PROBLEM 

N EQU 24 

M EQU 11 

X ORIG *+N 

OH ENT1 N-l 


1 Set each cell to the 

STZ X+N 

[-1 


1 number of the next 

ST1 X-1,1 

N 

— 1 in the sequence. 

DEC1 1 


N 

- 1 

J1P *-2 


N 

- 1 


man 
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ENTA 

1 

1 


(Now rll = 0) 

ENT2 

M-2 

N — l 


(Assume M > 2) 

LD1 

X,1 

(M — 2) (AT — 

1) 

Count around 

DEC2 

1 

(M — 2) (IV — 

1) 

the circle. 

J2P 

*-2 

(M — 2) (IV — 

1) 


LD2 

X,1 

N - 1 


rll = lucky man 

LD3 

X , 2 

N- 1 


rI2 = doomed man 

CHAR 


N - 1 


rI3 = next man 

STX 

X, 2(4:5) 

N- 1 


Store execution number. 

NUM 


IV — 1 



INCA 

1 

N - 1 



ST3 

x,i 

N — 1 


Take man from circle. 

ENT1 

0,3 

N- 1 



CMPA 

=N= 

IV- 1 



JL 

IB 

IV- 1 



CHAR 


1 


One man left; 

STX 

X, 1(4:5) 

1 


he is clobbered too. 

OUT 

X(18) 

1 


Print the answer. 

HLT 


1 



END 

OB 



1 


The last man is in position 15. The total time before output is (4(7V-l)(M+7.5) + 16)ii. 
Several improvements are possible, such as D. Ingalls’s suggestion to have three-word 
packets of code “DEC2 1; J2P NEXT; JMP OUT”, where OUT modifies the NEXT field so as 
to delete a packet. An asymptotically faster method appears in exercise 5. 1.1-5. 

SECTION 1.3.3 

1. (1 2 4)(3 6 5). 

2. a c, c o /; b <-> d. The generalization to arbitrary permutations is clear. 
fabcdef\ 

' \d b f c a e )' 

4. (adcfe). 

5. 12. (See exercise 20.) 

6 . The total time decreases by Hu for every blank word following a because lines 
30 32 cost 4 u while lines 26-28, 33-34, 36-38 cost 12u. It decreases by 2u for every 
blank word following a name, because lines 68-71 cost 5 u while 42 46 or 75-79 cost 7 u. 
Initial blanks and blanks between cycles do not affect the execution time. The position 
of blanks has no effect whatever on Program B. 

X = 2, I = 29, M = 5, N = 7, 17 = 3, V = 1. Total, by Eq. (18), 2161u. 

8. Yes; we would then keep the inverse of the permutation, so that x, goes to x : if 
and only if T[j] = i. (The final cycle form would then be constructed from right to 
left, using the T table.) 

9. No. For example, given (6) as input, Program A will produce “(ADG) (CEB)” as 
output, while Program B produces “(CEB) (DGA)”. The answers are equivalent but not 
identical, due to the nonuniqueness of cycle notation. The first element chosen for a 
cycle is the leftmost available name, in the case of Program A, and the last available 
distinct name to be encountered from right to left, in Program B. 
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10. (1) Kirchhoff’s law yields A = l + C— D\ B = A + J + P — 1; C = B — (P — L ); 
E = D — L\ G = E\ Q = Z\ W = S. (2) Interpretations: B = number of words of 
input = 16X — 1; C = number of nonblank words = Y\ D = C — M\ E = D — M; 
F = number of comparisons in names table search; H = N; K = M; Q = N; R = U: 
S = R — V; T = N — V since each of the other names gets tagged. (3) Summing up, 
we have (AF + 16 Y + 80X + 211V — 19 M + 9U — 1 QV)u, which is somewhat better than 
Program A because F is certainly less than 16AW. The time in the stated case is 983u, 
since F = 74. 

11. “Reflect’' it. For example, the inverse of (a c f)(b d) is ( db)(fca ). 

12. (a) The value in cell L+mn-1 is fixed by the transposition, so we may omit it from 
consideration. Otherwise if x = n(i — 1) + (j — 1) < mn — 1, the value in L + x should 
go to cell L + mx mod N = L + (mn{i — 1) + m(j — 1)) mod N = L + m(j — 1) + (i — 1), 
since mn = 1 (modulo N) and 0 < m(j — 1) + (i — 1) < N. (b) If one bit in each 
memory cell is available (for example, the sign), we can “tag” elements as we move 
them, using an algorithm like Algorithm I. [See M. F. Berman, JACM 5 (1958), 383- 
384.] If there is no room for a tag bit, tag bits can be kept in an auxiliary table, or 
else a list of representatives of all non-singleton cycles can be used: For each divisor 
d of N, we can transpose those elements that are multiples of d separately, since m is 
prime to N. The length of the cycle containing x, when gcd(a;, N) = d, is the smallest 
integer r > 0 such that m r = 1 (modulo N/d ). For each d, we want to find ip(N/d)/r 
representatives, one from each of these cycles. Some number-theoretic methods are 
available for this purpose, but they are not simple enough to be really satisfactory. 
An efficient but rather complicated algorithm can be obtained by combining number 
theory with a small table of tag bits. [See N. Brenner, CACM 16 (1973), 692-694.] 
Finally, there is a method analogous to Algorithm J; it is slower, but needs no auxiliary 
memory, and it performs any desired permutation in situ. [See P. F. Windley, Comp. J. 
2 (1959), 47-48; D. E. Knuth, Proc. IFIP Congress (1971), 1, 19-27; E. G. Cate and 
D. W. Twigg, ACM Trans. Math. Software 3 (1977), 104-110; F. E. Fic.h, J. I. Munro, 
and P. V. Poblete, SICOMP 24 (1995), 266-278.] 

13. Show by induction that, at the beginning of step J2, X[i] = +j if and only if 
j > m and j goes to i under 7r; X[i] = —j if and only if i goes to j under 7r fc+1 , where 
k is the smallest nonnegative integer such that TT k takes i into a number < m. 

14. Writing the inverse of the given permutation in canonical cycle form and dropping 
parentheses, the quantity A — N is the sum of the number of consecutive elements 
greater than a given element and immediately to its right. For example, if the original 
permutation is (1 6 5)(3 7 8 4), the canonical form of the inverse is (3 4 8 7)(2)(1 5 6); 
set up the array 

34872156 


• • • 

and the quantity A is the number of “dots,” 16. The number of dots below the fcth 
element is the number of right-to-left minima in the first k elements (there are 3 dots 
below 7 in the example above, since there are 3 right-to-left minima in 3487). Hence 
the average is Hi + H 2 + ■ • ■ + H n = (n + 1 )H n - n. 

15. If the first character of the linear representation is 1, the last character of the 
canonical representation is 1. If the first character of the linear representation is m > 1, 
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then “. . . 1 m . . . ” appears in the canonical representation. So the only solution is the 
permutation of a single object. (Well, there’s also the permutation of no objects.) 

16. 1324, 4231, 3214, 4213, 2143, 3412, 2413, 1243, 3421, 1324, .... 

17. (a) The probability that the cycle is an m-cycle is n\/m divided by nl H n , so 
Pm = 1 /(mH n ). The average length is pi + 2p 2 + 3p 3 + • • • = £m=i m/{mH n ) = n/H n . 
(b) Since the total number of m-cycles is /v. ! fin. the total number of appearances of 
elements in m-cycles is n\. Each element appears as often as any other, by symmetry, 
so k appears n\/n times in m-cycles. In this case, therefore, p m = 1/n for all k and m; 
the average is l m / n = (n + l)/2. 

18. See exercise 22(e). 

19. |.Pno — n\/e\ = n!/(n + 1)! — n!/(n + 2)! + • • • , an alternating series of decreasing 
magnitudes, which is less than n\/(n+ 1)! < I. 

20. There are «i + a 2 + • • • cycles in all, which can be permuted among one another, 
and each m-cycle can be independently written in m ways. So the answer is 

(«i + c*2 + •••)! 1 Q1 2“ 2 3 Q3 . . . . 

21. l/(ai! l Ql a2! 2“ 2 . . . ) if n = au + 2a 2 + • • • ; zero otherwise. 

Proof. Write out au 1-cycles, <22 2-cycles, etc., in a row, with empty positions; for 
example if a\ = 1, a 2 = 2, 03 = a 4 = • • • = 0, we would have “(-) ( — ) ( — )”. Fill 
the empty positions in all n! possible ways; we obtain each permutation of the desired 
form exactly ai! l ai a 2 ! 2“ 2 . . . times. 

22. (a) If fci + 2k 2 + • • • = n, the probability in (ii) is 17; >0 f( w i j, kj), which is assumed 
to equal (1 - w)w n /(k 1 \ l fcl fc 2 ! 2* 2 . . . ); hence 


= ( n /(»■«.))" n/<». a % + 1.) = 


j> 0 


j> 0 


m(km + 1) ' 


Therefore by induction 


f(w,m,k) 


1 / w m \ k 


k\ \ m 


f(w,m, 0). 


Condition (i) now implies that 

1 fw m \ k 

f(w, m, k) = 7T ( — ) e '“ • 

k\ \ m J 

[In other words, a m is chosen with a Poisson distribution; see exercise 1.2.10-15.] 

5Z P( n \ ki, k.2, . ■ ■) 

fci+2fc2H =n \j > 0 / fc,-l-C>fco-l — r. 

fci 


fcl +2&2H = n 

fel,fe2i'->0 


= (1 — w)i 


Hence the probability that on + 2a 2 H < n is (1 - w)(l + w 4 f w n ) = 1 - w n+1 . 

(c) The average of (f> is 


E E <t>(ki,k 2 , . . .) Pr (ai = k lt a 2 = k 2 , . . . ) 

0 ' fci+2fc2H —71 


(l-«;)^u; n ( <^(fci,fc2,...)/fci!l fcl fc 2 !2' :2 ...y 

n >0 ''fci-f2fc2H — =n / 
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(d) Let 4>{a\, 0 : 2 , . . . ) = 012 + <24 + ote + ■ ■ ■ . The average value of the linear 
combination 4> is the sum of the average values of <22, <24, ote, . . . ; and the average value 

fc>0 Jc>l v ' 

Therefore the average value of <f> is 

2 


2 + 4 + 6 + '" 2 


(H\w + H\w + H 2 W + H 2 W 5 + H 3 W 6 + • • • ). 


The desired answer is \Hy n / 2 \. 

(e) Set <j>(ai ,a 2 ,...) = z° m , and observe that the average value of 4> is 

. s k 




— iu m /m w Tn (z- 

e — e v 


l)/m _ y^ / g ~ 1 V 

^ 7! V m / 

j>0 J 


= <—>?*■( E 

n>0 ' 0<j<n/m 

= (1 - w) w n G nrn (z). 


Hence 


G nm ( 2 )- £ m )i p " fcm - m * fe! X) 


(— 1/m.y 


OKjKn/m 


m k k\ ^ 7 ! 

0< ji < n/m — fc 


the statistics are (min 0, ave 1/m, max [n/mj, dev \JTjm), when n > 2?n. 

23. The constant A is J’ 0 °° exp(— f — Ei(t)) dt, where E\(x) = e~ f dt/t. See Trans. 

Amer. Math. Soc. 121 (1966), 340-357, where many other results are proved, in par- 
ticular that the average length of the shortest cycle is approximately e 1 ln?r. Further 
terms of the asymptotic representation of l n have been found by Xavier Gourdon [Ph.D. 
thesis, Ecole Polytechnique (Paris, 1996)]; the series begins 


A n + |A — 


_1 +( 5 ^- K - l ) n )^~ 2 + (^ + !(-!)» + | 


■+x 


where w = e 2nt ^ 3 . William C. Mitchell has calculated a high-precision value of A = 
.62432 99885 43550 87099 29363 83100 83724 41796+ [Math. Comp. 22 (1968), 411- 
415]; no relation between A and classical mathematical constants is known. The same 
constant had. however, been computed in another context by Karl Dickman in Arkiv 
for Mat., Astron. och Fys. 22A, 10 (1930), 1-14; the coincidence wasn’t noticed until 
many years later [Theor. Comp. Sci. 3 (1976), 373]. 

24. See D. E. Knuth, Proc. IFIP Congress (1971), 1, 19-27. 

25. One proof, by induction on N, is based on the fact that when the Nth element is 
a member of s of the sets it contributes exactly 


(0) - (0 + (2) = (! - I)" = ^0 

to the sum. Another proof, by induction on M, is based on the fact that the number 
of elements that are in Sm but not in Si U • • • U Sm- 1 is 


|Sm|- I Sj n Sm ! + \Sjns k rs M \ . 

l<j<M 1 <j<k<M 
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26. Let No = N and let 


N * = |Sji n • ■ ■ n Sj J . 

1 <h< -<Jk<M 

Then the desired formula is 

MTh+ i+ (TK 2 ---- 

This may be proved from the principle of inclusion and exclusion itself, or by using the 
formula 


oo-mi;,)- 

as in exercise 25. 



5 


sr 


27. Let Sj be the multiples of m 3 in the stated range and let N = am i Then 

| Sj n Sk | = N/mjmk, etc., so the answer is 


iv-jv r -+jv v 

t J 772 ■ ' v 

l<j<t 3 1 <j<k<t 


1 

mjm k 


••• = N 



This also solves exercise 1.2.4—30, if we let mi , . . . , mt be the primes dividing N. 

29. When passing over a man, assign him a new number (starting with n + 1). Then 
the fcth man executed is number 2k, and man number j for j > n was previously 
number (2 j) mod (2 n + 1). 

31. See CMath, Section 3.3. When m = 2, the fcth man executed is number 2 n + 
1 - (2 n + 1 - 2fc)2 l - lg(2n / (2n+1-2i:,) J . [Armin Shams, Proc. Nat. Computer Conf. 2002 , 
English papers section, 2 (Mashhad, Iran: Ferdowsi University, 2002), 29-33.] 

32. (a) In fact, k — 1 < ttj, < fc + 2 when k is even; k — 2 < 7T/t < k + 1 when k is odd. 
(b) Choose the exponents from left to right, setting e k = 1 if and only if k and k + 1 are 
in different cycles of the permutation so far. [Steven Alpern, J. Combinatorial Theory 
B25 (1978), 62-73.] 

33. For l = 0, let (a 0 i, £*02; An, /802) = (7r,p;e,e) and (an, Q12; /3n , /I12) = (e,e;n,p), 
where 7r = (1 4) (2 3), p = (15)(2 4), and e = (). 

Suppose we have made such a construction for some l > 0, where o? jk = 0 3k = () 
for 0 < j < m and 1 < k < n. Then the permutations 


(A(jm+j') li ■ • ■ t A(jm+j')(4n ) ! B(jra+j') 1> ■ ■ ■ i B(jm+j')(4n)) — 

(a otj 1 ct, . . ,,a~otj n a, t~ O ij’iT, . . . ,t~ oij, n T, 
a p jn a, ...,a 0j\cr, r fiji n T,...,T Pj'\T\ 

(?~ PjiO, . . ,,G~Pj n O, T~ Pji\T, . . .,T~Pj' n T, 

a Otj n (T, . . . ,G~ ctjicr, T~atji n T , . . . ,T~aj>iT ) 

have the property that 

' ' • ^(im+i')(4n)-B(jm+j')(4n) = 

tj _ (1234 5)(Tr - (12345)r(T _ (5 4321)CTr _ (54321)r 

if i = j and i! = j 1 , otherwise the product is (). Choosing a = (2 3) (4 5) and r = (3 4 5) 
will make the product (12 3 4 5) as desired, when im + i’ = jm + j’ . 

The construction that leads from l to 1 + 1 is due to David A. Barrington [J. Comp. 
Syst. Sci. 38 (1989), 150-164], who proved a general theorem by which any Boolean 
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function can be represented as a product of permutations of {1,2, 3, 4, 5}. With a 
similar construction we can, for example, find sequences of permutations (aji, . . . , otj n ‘, 
/Iji, . . . ,/3 ,„) such that 


&il(3jlO'i2l3j2 • • - Qinfijn — 


(12345), if i<j- 

0, if i > r, 


for 0 < i,j < m = 2 2 when n = 6 (+1 — 4 i+1 . 

34. Let N = to + n. If m _L n there is only one cycle, because every element can be 
written in the form am mod N for some integer a. And in general if d = gcd (m, n), 
there are exactly d cycles Co, C i, . . . , Cd- i, where C, contains the elements {j. j + d, 

. . . ,j + N — d] in some order. To carry out the permutation, we can therefore proceed 
as follows for 0 < j < d (in parallel, if convenient): Set t t— Xj and A j; then 
while ( k + m) mod N / j , set x <— X( k+m ) mod n and k ( k + m) mod N\ finally 
set Xk <— t. In this algorithm the relation (k + to) mod IV / j will hold if and only if 
( k + to) mod N > d, so we can use whichever test is more efficient. [W. Fletcher and 
R. Silver, CACM 9 (1966), 326.] 

35. Let M = l + m + n and N = l + 2 to. + n. The cycles for the desired rearrangement 
are obtained from the cycles of the permutation on {0, 1, . . . , N — 1} that takes k to 
(k + l + m.) mod N, by simply striking out all elements of each cycle that are > M. 
(Compare this behavior with the similar situation in exercise 29.) Proof: When the 
hinted interchange sets Xk «— xy and xy t— xy for some k with A:' = ( k + l + m ) mod N 
and A" = (A' + l + to) mod N and A' > M, we know that xy = xy\ hence the 
rearrangement a/3 7 — > y/Ia replaces Xk by xy . 

It follows that there are exactly d = gcd (/ + to, to + n) cycles, and we can use an 
algorithm similar to the one in the previous exercise. 

A slightly simpler way to reduce this problem to the special case in exercise 34 is 
also noteworthy, although it makes a few more references to memory: Suppose 7 = r y'^" 
where j'y” | = |a|. Then we can change to -y" P'y'a, and interchange 7 "(3 with 

7'. A similar approach works if |a| > |7| . [See J. L. Mohammed and C. S. Subi, 
J. Algorithms 8 (1987), 113-121.] 


SECTION 1.4.1 

1. Calling sequence: JMP MAXN; or, JMP MAX100 if n = 100. 

Entry conditions: For the MAXN entrance, rI3 = n; assume n > 1. 

Exit conditions: Same as in (4). 

2. MAX50 STJ EXIT 

ENT3 50 
JMP 2F 

3. Entry conditions: n = rll if rll > 0; otherwise n = 1. 

Exit conditions: rA and rI2 as in (4); rll unchanged; rI3 = min(0,rll); rj = 
EXIT + 1; Cl unchanged if n = 1, otherwise Cl is greater, equal, 
or less, according as the maximum is greater than A'[l], equal 
to X[l], and rI2 > 1, or equal to A'[l] with rI2 = 1. 

(The analogous exercise for (9) would of course be somewhat more complicated.) 


4. SMAX1 

ENT1 

1 

r = 1 

SMAX 

STJ 

EXIT 

general r 


JMP 

2F 

continue as before 
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DEC3 

0,1 

decrease by r 

J3P 

IB 


EXIT JMP 

* 

exit. 

Calling sequence: 

JMP 

SMAX; or, JMP SMAX1 if r = 1 


Entry conditions: rI3 = n, assumed positive; for the SMAX entrance, rll = r, assumed 
positive. 

Exit conditions: rA = max 0 < fc < n/r CONTENTS (X + n - kr) = CONTENTS (X + rI2); and 
rI3 = (n - 1 ) mod r + 1 - r = -((- n ) mod r). 

5. Any other register can be used. For example, 

Calling sequence: ENTA *+2 

JMP MAX100 
Entry conditions: None. 

Exit conditions: Same as in ( 4 ). 

The code is like ( 1 ), but the first instruction becomes “MAX100 STA EXIT(0:2)”. 

6 . (Solution by Joel Goldberg and Roger M. Aarons.) 


MOVE 

STJ 

3F 



STA 

4F 

Save rA and rI2. 


ST2 

5F (0 : 2) 



LD2 

3F ( 0 : 2 ) 

rI2 +- address of “NOP A, 1(F)”. 


LDA 

0, 2(0:3) 

rA <- “A, I”. 


STA 

*+2(0:3) 



LD2 

5F(0 : 2) 

Restore rI2, because I might be 2 


ENTA 

* 

rA <— indexed address. 


LD2 

3F(0 : 2) 



LD2N 

0, 2(4:4) 

rI2 < F. 


J2Z 

IF 



DECA 

0,2 



STA 

2F(0 : 2) 



DEC1 

0,2 

rll «- rll + F. 


ST1 

6 F ( 0 : 2 ) 


2H 

LDA 

*,2 


6 H 

STA 

* ,2 



INC2 

1 

Increase rI2 until it becomes zero. 


J2N 

2B 


1H 

LDA 

4F 

Restore rA and rI2. 

5H 

ENT 2 

* 


3H 

JMP 

* 

Exit to the NOP instruction. 

4H 

CON 

0 1 



7. (1) An operating system can allocate high-speed memory more efficiently if pro- 
gram blocks are known to be “read-only.” (2) An instruction cache in hardware will be 
faster and less expensive if instructions cannot change. (3) Same as (2), with “pipeline” 
in place of “cache.” If an instruction is modified after entering a pipeline, the pipeline 
needs to be flushed; the circuitry needed to check this condition is complex and time- 
consuming. (4) Self- modifying code cannot be used by more than one process at once. 
(5) Self-modifying code can defeat a jump-trace routine (exercise 1.4. 3. 2-7), which is 
an important diagnostic tool for “profiling” (that is, for computing the number of times 
each instruction is executed). 
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SECTION 1.4.2 

1. If one coroutine calls the other only once, it is nothing but a subroutine; so we 
need an application in which each coroutine calls the other in at least two distinct 
places. Even then, it is often easy to set some sort of switch or to use some property 
of the data, so that upon entry to a fixed place within one coroutine it is possible to 
branch to one of two desired places; again, nothing more than a subroutine would be 
required. Coroutines become correspondingly more useful as the number of references 
between them grows larger. 

2. The first character found by IN would be lost. [We started OUT first because lines 
58- 59 do the necessary initialization for IN. If we wanted to start IN first, we’d have 
to initialize OUT by saying “ENT4 -16”, and clearing the output buffer if it isn’t known 
to be blank. Then we could make line 62 jump first to line 39.] 

3. Almost true, since “CMPA =10=” within IN is then the only comparison instruction 
of the program, and since the code for is 40. (!) But the comparison indicator isn’t 
initialized; and if the final period is preceded by a replication digit, it won’t be noticed. 
[Note: The most nitpickingly efficient program would probably remove lines 40, 44, and 
48, and would insert “CMPA PERIOD” between lines 26 and 27, “CMPX PERIOD” between 
lines 59 and 60. The state of the comparison indicator should then become part of the 
coroutine characteristics in the program documentation.] 

4. Here are examples from three rather different computers of historic importance: 
(i) On the IBM 650, using SOAP assembly language, we would have the calling se- 
quences “LDD A” and “LDD B”, and linkage “A STD BX AX” and “B STD AX BX” 
(with the two linkage instructions preferably in core), (ii) On the IBM 709, using 
common assembly languages, the calling sequences would be “TSX A , 4” and “TSX B , 4” ; 
the linkage instructions would be as follows: 


SXA 

BX,4 

B SXA 

AX ,4 

AXT 

1-A1.4 

BX AXT 

1-B1.4 

TRA 

1,4 

TRA 

1,4 


(iii) On the CDC 1604, the calling sequences would be “return jump” (SLJ 4) to A or B, 
and the linkage would be, for example, 

A: SLJ Bl; ALS 0 
B: SLJ Al; SLJ A 

in two consecutive 48-bit words. 

5. "STA H0LDAIN; LDA H0LDA0UT” between OUT and 0UTX, and “STA H0LDA0UT; LDA 
H0LDAIN" between IN and INX. 

6. Within A write ”JMP AB” to activate B, “JMP AC” to activate C. Locations BA, BC, 
CA. and CB would, similarly, be used within B and C. The linkage is: 


AB 

STJ 

AX 

BC 

STJ 

BX 

CA 

STJ 

CX 

BX 

JMP 

Bl 

CX 

JMP 

Cl 

AX 

JMP 

Al 

CB 

STJ 

CX 

AC 

STJ 

AX 

BA 

STJ 

BX 


JMP 

BX 


JMP 

CX 


JMP 

AX 


[Note: With n coroutines, 2(n — l)n cells would be required for this style of linkage. If n 
is large, a “centralized” routine for linkage could of course be used; a method with 371+2 
cells is not hard to invent. But in practice the faster method above requires just 2m 
cells, where m is the number of pairs (i,j) such that coroutine i jumps to coroutine j. 
\\ hen there are many coroutines each independently jumping to others, the sequence 
of control is usually under external influence, as discussed in Section 2.2.5.] 
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SECTION 1.4. 3.1 

1. FCHECK is used only twice, both times immediately followed by a call on MEMORY. 
So it would be slightly more efficient to make FCHECK a special entrance to the MEMORY 
subroutine, and also to make it put — R in rI2. 


SHIFT 

J5N 

ADDRERR0R 

3. MOVE J3Z 

CYCLE 


DEC3 

5 

JMP 

MEMORY 


J3P 

FERR0R 

SRAX 

5 


LDA 

AREG 

LDI 

11 REG 


LDX 

XREG 

LDA 

SIGN1 


LDI 

1F,3(4:5) 

JAP 

*+3 


ST1 

2F (4 : 5) 

J1NZ 

MEMERR0R 


J5Z 

CYCLE 

STZ 

SIGN1 (0 : 0) 

2H 

SLA 

1 

CMP1 

=BEGIN= 


DEC5 

1 

JGE 

MEMERR0R 


J5P 

2B 

STX 

0,1 


JMP 

ST0REAX 

LDA 

CLOCK 


SLA 

1 

INCA 

2 


SRA 

1 

STA 

CLOCK 


SLAX 

1 

INC1 

1 


SRAX 

1 

ST1 

11 REG 


SLC 

1 

INC5 

1 

1H 

SRC 

1 1 

DEC3 

1 




JMP 

MOVE | 


4. Just insert “IN 0(16)” and “JBUS *(16)” between lines 003 and 004. (Of course 
on another computer this would be considerably different since it would be necessary 
to convert to MIX character code.) 

5. Central control time is 34m, plus 15u if indexing is required; the GETV subroutine 
takes 52m, plus 5m if L / 0; extra time to do the actual loading is 11m for LDA or LDX, 
13m for LDi, 21m for ENTA or ENTX, 23m for ENTi (add 2m to the latter two times if 
M = 0). Summing up, we have a total time of 97 u for LDA and 55m for ENTA, plus 15m 
for indexing, and plus 5m or 2m in certain other circumstances. It would seem that 
simulation in this case is causing roughly a 50:1 ratio in speeds. (Results of a test run 
that involved 178m of simulated time required 8422m of actual time, a 47:1 ratio.) 

7 . Execution of IN or OUT sets a variable associated with the appropriate input device 
to the time when transmission is desired. The “CYCLE” control routine interrogates 
these variables on each cycle, to see if CLOCK has exceeded either (or both) of them; 
if so, the transmission is carried out and the variable is set to oo. (When more than 
two I/O units must be handled in this way, there might be so many variables that it 
would be preferable to keep them in a sorted list using linked memory techniques; see 
Section 2.2.5.) We must be careful to complete the I/O when simulating HLT. 

8. False; rI6 can equal BEGIN, if we “fall through” from the location BEGIN - 1. But 
then a MEMERR0R will occur, trying to STZ into TIME! On the other hand, we always do 
have 0 < rI6 < BEGIN, because of line 254. 

SECTION 1.4. 3. 2 

1. Change lines 48 and 49 to the following sequence: 
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XREG 

ORIG 

*+2 


JMP 

-1,1 

LEAVE 

STX 

XREG 

1H 

JMP 

*+l 


ST1 

XREG+1 


STA 

-1,1 


LD1 

JREG (0 : 2) 


LD1 

XREG+1 


LDA 

-1,1 


LDX 

XREG 


LDX 

IF 


LDA 

AREG 


STX 

-1,1 

LEAVEX 

JSJ 

* 

operator "JSJ” here is, of course, 

particularly crucial. 


* TRACE ROUTINE 


STA 

BUF+1, 1(4: 5) 


ORIG 

*+99 


ENTA 

8 

BUF 

CON 

0 


JN0V 

IF 



. . .lines 02-04 


ADD 

BIG 


ST1 

11 REG 

1H 

JL 

IF 



. . .lines 05-07 


INCA 

1 

PTR 

ENT1 

-100 


JE 

IF 


JBUS 

*(0) 


INCA 

1 


STA 

BUF+1 ,1(0:2) 

1H 

STA 

BUF+1, 1(3: 3) 



. . .lines 08-11 


INC1 

10 


STA 

BUF +2 , 1 


JIN 

IF 



. . .lines 12-13 


OUT 

BUF-99 (0) 


LDA 

AREG 


ENT1 

-100 


STA 

BUF+3,1 

1H 

ST1 

PTR(0 : 2) 


LDA 

11 REG 



lines 14-31 


STA 

BUF+4,1 


LD1 

I1REG 


ST2 

BUF+5 , 1 



lines 32-35 


ST3 

BUF+6 , 1 


ST1 

11 REG 


ST4 

BUF+7 , 1 



lines 36-48 


ST5 

BUF +8 , 1 


LD1 

11 REG 


ST6 

BUF+9 , 1 



lines 49-50 


STX 

BUF+10,1 

B4 

EQU 

1(1:1) 


LDA 

JREG (0:2) 

BIG CON 

B4-8 ,B4-1 (1 : 1) 


A supplementary routine that writes out the final buffer and rewinds tape 0 should 
be called after all tracing has been performed. 

3. Tape is faster; and the editing of this information into characters while tracing 
would consume far too much space. Furthermore the tape contents can be selectively 
printed. 

4. A true trace, as desired in exercise 6, would not be obtained, since restriction (a) 
mentioned in the text is violated. The first attempt to trace CYCLE would cause a loop 
back to tracing ENTER+1, because PREG is clobbered. 

6. Suggestion: Keep a table of values of each memory location within the trace area 
that has been changed by the outer program. 

7. The routine should scan the program until finding the first jump (or conditional 
jump) instruction; after modifying that instruction and the one following, it should 
restore registers and allow the program to execute all its instructions up to that point, 
in one burst. [This technique can fail if the program modifies its own jump instructions, 
or changes non-jumps into jumps. For practical purposes we can outlaw such practices, 
except for STJ, which we probably ought to handle separately anyway.] 
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1. (a) No; the input operation might not yet be complete, (b) No; the input operation 
might be going just a little faster than the HOVE. This proposal is much too risky. 


2. ENT1 2000 

JBUS * (6) 

MOVE 1000(50) 


MOVE 1050(50) 


OUT 2000(6) 

1 


3. W0RD0UT STJ 

IF 

DEC5 100 

STA 

0,5 

JMP 2B 

INC5 

1 

BUFMAX CON ENDBUF1 

2H CMP 5 

BUFMAX 

* BUFFER AREAS 

1H JNE 

* 

OUTBUFl ORIG *+100 

OUT 

-100, 5(V) 

ENDBUF1 CON ENDBUF2 

LD5 

0,5 

0UTBUF2 ORIG *+100 

ST5 

BUFMAX 

ENDBUF2 CON ENDBUF1 1 

At the beginning of the program, give the instruction “ENT5 OUTBUFl”. At 

the program, say 

LDA 

BUFMAX 

INC5 1 

DECA 

100,5 

CMP5 BUFMAX 

JAZ 

*+6 

JNE *-3 

STZ 

0,5 

OUT -100, 5 (V) 


4. If the calculation time exactly equals the I/O time (which is the most favorable 
situation), both the computer and the peripheral device running simultaneously will 
take half as long as if they ran separately. Formally, let C be the calculation time for 
the entire program, and let T be the total I/O time required; then the best possible 
running time with buffering is max(C, T), while the running time without buffering is 
C + T; and of course \ (C + T) < max((7, T) < C + T. 

However, some devices have a “shutdown penalty” that causes an extra amount 
of time to be lost if too long an interval occurs between references to that unit; in such 
a case, better than 2:1 ratios are possible. (See, for example, exercise 19.) 

5. The best ratio is ( n + 1):1. 


6. ( IN 

INBUF1 (U) \ 


r in 

INBUF2 (U)l 

IENT6 

INBUF2+99J 


1ENT6 

INBUF1+99J 


(possibly preceded by IOC 0(U) to rewind the tape just in case it is necessary). 


7. One way is to use coroutines: 


INBUF1 

ORIG 

*+100 


INC6 

1 

INBUF2 

ORIG 

*+100 


J6N 

2B 

1H 

LDA 

INBUF2+100.6 


IN 

INBUFl(U) 


JMP 

MAIN 


ENN6 

100 


INC6 

1 


JMP 

IB 


J6N 

IB 

W0RDIN 

STJ 

MAINX 

W0RDIN1 

IN 

INBUF2(U) 

WORD I NX 

JMP 

WORD INI 


ENN6 

100 

MAIN 

STJ 

W0RDINX 

2H 

LDA 

INBUFl+100,6 

MAINX 

JMP 

* 1 


JMP 

MAIN 
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Adding a few more instructions to take advantage of special cases will actually make 
this routine faster than (4). 

8. At the time shown in Fig. 23, the two red buffers have been filled with line images, 
and the one indicated by NEXTR is being printed. At the same time, the program is 
computing between RELEASE and ASSIGN. When the program ASSIGNS, the green buffer 
indicated by NEXTG becomes yellow; NEXTG moves clockwise and the program begins to 
fill the yellow buffer. When the output operation is complete, NEXTR moves clockwise, 
the buffer that has just been printed turns green, and the remaining red buffer begins 
to be printed. Finally, the program RELEASES the yellow buffer and it too is ready for 
subsequent printing. 


9, 10 

, 11- 



time 

action (N = 1) 

action (N = 2) 

action (N = 4) 

0 

ASSIGN(BUFl) 

ASSIGN(BUFl) 

ASSIGN (BUF1) 

1000 

RELEASE, OUT BUF1 

RELEASE, OUT BUF1 

RELEASE, OUT BUF1 

2000 

ASSIGN (wait) 

ASSIGN (BUF2) 

ASSIGN (BUF2) 

3000 


RELEASE 

RELEASE 

4000 


ASSIGN (wait) 

ASSIGN(BUF3) 

5000 



RELEASE 

6000 



ASSIGN (BUF4) 

7000 



RELEASE 

8000 



ASSIGN (wait) 

8500 

BUF1 assigned, output stops 

BUF1 assigned, OUT BUF2 

BUF1 assigned, OUT BUF2 

9500 

RELEASE, OUT BUF1 

RELEASE 


10500 

ASSIGN (wait) 

ASSIGN (wait) 


15500 



RELEASE 


and so on. Total time when N = 1 is llOOOOu; when N = 2 it is 89000 a; when N = 3 
it is 81500u; and when N > 4 it is 76000u. 

12. Replace the last three lines of Program B by 

STA 2F 
LDA 3F 
CMP A 15,5(5:5) 

LDA 2F 
LD5 -1,5 
DEC6 1 
JNE IB 
JMP COMPUTE 

JMP *-l (or JMP COMPUTEX) 

2H CON 0 

3H ALF uuuu • I 

13. JRED CONTROL (U) 

J6NZ *-l | 

14. If N = 1 the algorithm breaks down (possibly referring to the buffer while I/O is 
in progress); otherwise the construction will have the effect that there are two yellow 
buffers. This can be useful if the computational program wants to refer to two buffers at 
once, although it ties up buffer space. In general, the excess of ASSIGNS over RELEASES 
should be nonnegative and not greater than N. 
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u 

EQU 

0 

IN 

V 

EQU 

1 

OUT 

BUF1 

0RIG 

*+100 

IN 

BUF2 

0RIG 

*+100 

OUT 

BUF3 

0RIG 

*+100 

DEC1 

TAPECPY 

IN 

BUFl(U) 

J1P 


ENT1 

99 

OUT 

1H 

IN 

BUF2(U) 

HLT 


OUT 

BUFl(V) 

END 


This is a special case of the algorithm indicated in Fig. 26. 


BUF3CU) 

BUF2CV) 

BUFl(U) 

BUF3(V) 

3 

IB 

BUFl(V) 
TAPECPY | 


18. Partial solution: In the algorithms below, t is a variable that equals 0 when the 
I/O device is idle, and 1 when it is active. 


Algorithm A (ASSIGN, a normal state subroutine). 

This algorithm is unchanged from Algorithm 1.4.4A. 

Algorithm R (RELEASE, a normal state subroutine). 

Rl. Increase n by one. 

R2. If I. = 0, force an interrupt, going to step B3 (using the INT operator). | 

Algorithm B (Buffer control routine, which processes interrupts). 

Bl. Restart the main program. 

B2. If n = 0, set t 4— 0 and go to Bl. 

B3. Set ft— 1, and initiate I/O from the buffer area specified by NEXTR. 

B4. Restart the main program; an “I/O complete” condition will interrupt it and 
lead to step B5. 

B5. Advance NEXTR to the next clockwise buffer. 

B6. Decrease n by one, and go to step B2. | 

19. If C < L we can have t k = ( k - 1 )L, u k = t k + T, and v k = u k + C if and only if 
NL >T + C. If C > L the situation is more complex; we can have u k = ( k - l)C + T 
and v k = kC + T if and only if there are integers aq < a 2 < • • • < a n such that 
tk (k — 1 )L + a k P satisfies u k — T > t. k > v k — n for N < k < n. An equivalent 
condition is that NC > b k for N < k < n, where b k = C + T+ ((k - 1)(C - L)) mod P. 
Let ci = max{6| + i, . . . , 6„ , 0}; then c/ decreases as l increases, and the smallest value 
of N that keeps the process going steadily is the minimum l such that c t /l < C. 
Since ci < C + T + P and c; < L + T + n(C — L), this value of N never exceeds 
| mm{C + T + P, L + T + n(C - L)}/C] . [See A. Itai and Y. Raz, CACM 31 (1988) 
1338-1342.] 

In the stated example we have therefore (a) N = 1; (b) N = 2; (c) N = 3. 
cjv = 2.5; (d) N = 35, c N = 51.5; (e) N = 51, c N = 101.5; (f) N = 41, c N = 102; 
(g) N = 11, c N = 109.5; (h) N = 3, cjv = 149.5; (i) N = 2, c N = 298.5. 


SECTION 2.1 

1. (a) SUIT (NEXT (TOP) ) = SUIT(NEXT(242) ) = SUIT(386) = 4. (b) A. 

2. Whenever V is a link variable (else CONTENTS (V) makes no sense) whose value is 
not A. It is wise to avoid using L0C in contexts like this. 

3. Set NEWCARD 4- TOP, and if TOP ^ A set TOP 4- NEXT (TOP) . 
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4. Cl. Set X 4— LOC(TOP). (For convenience we make the reasonable assumption 

that TOP = NEXT (LOC (TOP) ) , namely that the value of TOP appears in the 
NEXT field of the location where it is stored. This assumption is compatible 
with program (5), and it saves us the bother of writing a special routine for 
the case of an empty pile.) 

C2. If NEXT(X) / A, set X 4— NEXT(X) and repeat this step. 

C3. Set NEXT (X) 4- NEWCARD, NEXT(NEWCARD) 4- A, TAG(NEWCARD) 4- 1. | 

5. Dl. Set X 4— LOC(TOP), Y 4— TOP. (See step Cl above. By hypothesis, Y 7^ A. 

Throughout the algorithm that follows, X trails one step behind Y in the sense 
that Y = NEXT(X) .) 

D2. If NEXT (Y) # A, set X 4- Y, Y 4- NEXT(Y) , and repeat this step. 

D3. (Now NEXT(Y) = A, so Y points to the bottom card; also X points to the 
next-to-last card.) Set NEXT(X) 4- A, NEWCARD 4- Y. | 

6. Notations (b) and (d). Not (a)! CARD is a node, not a link to a node. 

7. Sequence (a) gives NEXT (LOC (TOP) ) , which in this case is identical to the value of 
TOP; sequence (b) is correct. There is no need for confusion; consider the analogous 
example when X is a numeric variable: To bring X into register A, we write LDA X, not 
ENTA X, since the latter brings LOC(X) into the register. 

8. Let rA ee N, rll = X. 


ENTA 0 

BL 

, N 4- 0. 

INCA 1 B3. N 4— N + : 

LD1 TOP X 4- TOP. 

LD1 0,1 (NEXT) X 4— NEXT(X) 

J1Z *+4 B2, 

. Is X = A? 

J1NZ *-2 | 

Let rI2 = 

: X. 



PRINTER 

EQU 

18 

Unit number for line printer 

TAG 

EQU 

1:1 


NEXT 

EQU 

4:5 

Definition of fields 

NAME 

EQU 

0:5 


PBUF 

ALF 

PILE 

Message printed in case 


ALF 

EMPTY 

pile is empty 


ORIG 

PBUF+24 


BEGIN 

LD2 

TOP 

Set X 4- TOP. 


J2Z 

2F 

Is the pile empty? 

1H 

LDA 

0,2(TAG) 

rA 4- TAG(X) . 


ENT1 

PBUF 

Get ready for MOVE instruction. 


JBUS 

* (PRINTER) 

Wait until printer is ready. 


JAZ 

*+3 

Is TAG = 0 (is card face up)? 


MOVE 

PAREN(3) 

No: Copy parentheses. 


JMP 

*+2 



MOVE 

BLANKS (3) 

Yes: Copy blanks. 


LDA 

1,2 (NAME) 

rA 4- NAME(X) . 


STA 

PBUF+1 



LD2 

0,2 (NEXT) 

Set X 4- NEXT (X) . 

2H 

OUT 

PBUF (PRINTER) 

Print the line. 


J2NZ 

IB 

If X 7! A, repeat the print loop. 

DONE 

HLT 
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PAREN 

ALF 

BLANKS 

ALF 


ALF 


ALF 

SECTION 

2.2.1 


1. Yes. (Consistently insert all items at one of the two ends.) 

2. To obtain 325641, do SSSXXSSXSXXX (in the notation of the following exercise). 
The order 154623 cannot be achieved, since 2 can precede 3 only if it is removed from 
the stack before 3 has been inserted. 

3. An admissible sequence is one in which the number of X’s never exceeds the number 
of S’s if we read from the left to the right. 

Two different admissible sequences must give a different result, since if the two 
sequences agree up to a point where one has S and the other has X, the latter sequence 
outputs a symbol that cannot possibly be output before the symbol just inserted by 
the S of the former sequence. 

4. This problem is equivalent to many other interesting problems, such as the enumer- 
ation of binary trees, the number of ways to insert parentheses into a formula, and the 
number of ways to divide a polygon into triangles, and it appeared as early as 1759 in 
notes by Euler and von Segner (see Section 2. 3.4.6). 

The following elegant solution uses a “reflection principle” due to D. Andre: There 
are obviously ( n ) sequences of S s and X’s that contain n of each. It remains to evaluate 
the number of inadmissible sequences (those that contain the right number of S’s and 
X’s but violate the other condition). In any inadmissible sequence, locate the first X 
for which the X’s outnumber the S’s. Then in the partial sequence leading up to and 
including this X, replace each X by S and each S by X. The result is a sequence with 
(n + 1) S s and ( n — 1) X’s. Conversely for every sequence of the latter type we can 
reverse the process and find the inadmissible sequence of the former type that leads to it. 
For example, the sequence XXSXSSSXXSSS must have come from SSXSXXXXXSSS. 
This correspondence shows that the number of inadmissible sequences is ( 2 " ). Hence 
a « = (n‘ ) - (n-i)- [Comptes Rendus Acad. Sci. 105 (Paris, 1887), 436-437.] 

Using the same idea, we can solve the more general “ballot problem” of probability 
theory, which essentially is the enumeration of all partial admissible sequences with a 
given number of S’s and X’s. This problem was actually resolved as early as 1708 
by Abraham de Moivre, who showed that the number of sequences containing l A’s 
and m B’s, and containing at least one initial substring with n more A’s than B’s, is 
/(/, m, n) = ( min |+7_ n) ). In particular, a n = ( 2 n n ) - f(n,n, 1) as above. (De Moivre 
stated this result without proof [Philos. Trans. 27 (1711), 262-263]; but it is clear from 
other passages in his paper that he knew how to prove it, since the formula is obviously 
true when l > m + n, and since his generating-function approach to similar problems 
yields the symmetry condition f(l,m,n) = f(m + n,l — n,n) by simple algebra.) For 
the later history of the ballot problem and some generalizations, see the comprehensive 
survey by D. E. Barton and C. L. Mallows, Annals of Math. Statistics 36 (1965), 
236-260; see also exercise 2.3.4.4-32 and Section 5.1.4. 

We present here a new method for solving the ballot problem with the use of 
double generating functions, since this method lends itself to the solution of more 
difficult, problems such as the question in exercise 11. 
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Let g nrn be the number of sequences of S’s and X’s of length n, in which the 
number of X’s never exceeds the number of S’s if we count from the left, and in which 
there are m more S’s than X’s in all. Then a n = <?( 2 n )o- Obviously g nm is zero unless 
m + n is even. We see easily that these numbers can be defined by the recurrence 
relations 


9(n+l)m 9n(m — l) T 9n(m + l) i m > 0, 71 ^ 0; QOm — ^Om • 

Consider the double generating function G(x,z) = m 9 nmX m z n , and let g(z) = 

G(0, z). The recurrence above is equivalent to the equation 


(x + G(x,z) = —g(z) + ~(G(x,z) - l), 
V x / x z 


G(x,z) 


zg{z) - X 

z(x 2 + 1) — x ' 


This equation unfortunately tells us nothing if we set x = 0, but we can proceed by 
factoring the denominator as a( 1 — ri(z)x)(l — r 2 (z)x) where 


ri(z) = ~(1 + \/l - 4 z 2 ), r 2 {z) - — (l - y/ 1 - 4 z 2 ). 

(Note that n + r 2 = 1/a; rir 2 = 1.) We now proceed heuristically; the problem is to 
find some value of g(z) such that G(x, z) as given by the formula above has an infinite 
power series expansion in x and a. The function ^(a) has a power series, and r2(0) = 0; 
moreover, for fixed a, the value x = r 2 (a) causes the denominator of G(x, a) to vanish. 
This suggests that we should choose g(z) so that the numerator also vanishes when 
x = r 2 (a); in other words, we probably ought to take zg(z) = r 2 (z). With this choice, 
the equation for G(x, z) simplifies to 


G(x, a) 


r 2 (z) 

z( 1 - r 2 (z)x) 


'^2{r 2 (z)) n+1 x n z \ 

n>0 


This is a power series expansion that satisfies the original equation, so we must have 
found the right function g(z). 

The coefficients of g(z ) are the solution to our problem. Actually we can go further 
and derive a simple form for all the coefficients of G(x, a): By the binomial theorem, 


">(*) = £ =“ + ‘( /‘) 


1 


2k + 1' 


Let w = a 2 and r 2 (z) = zf(w). Then f(w) = Y , k > o A k {l,-2)w k in the notation of 


exercise 1.2.6-25; hence 






/M r = 


-2)w k . 




k> 0 



We now have 





G(x. 

-*) = E 

Am(n + 1, 

— 2) x n z 2m+n . 



n,m 




so the general solution is 

_ / 

2n + 1 \ 

2m + 1 

/ 2n \ / 

2 n \ 

9(2n)(2m) — ( 

n — m ) 

2n + 1 

V 77. — 777. / \ 

, n — m — l) 

5(2n + l)(2m+l) = ^ 

277. -f- 2 \ 

2m + 2 

/2n+l\ / 

' 2n + l \ 

.77. — m J 

2tt. -|- 2 

\n — m ) ' 

\n — m — 1 / 
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^3 k and pj < p k , we must have taken pj off the stack before pi- was put on: if 
Pj > Pk, we must have left p k on the stack until after pj was put on. Combining these 
two rules, the condition i < j < k and pj < p *. < p x is impossible, since it means that 
Pj must go off before p k and after p;, yet p, appears after p k . 


Conversely, the desired permutation can be obtained by using the following algo- 
rithm: “For j = 1, 2, . . . , n, input zero or more items (as many as necessary) until pj 
first appears in the stack; then output pj.” This algorithm can fail only if we reach a 
j for which pj is not at the top of the stack but it is covered by some element p k for 
k > j. Since the values on the stack are always monotone increasing, we have pj < p k . 
And the element p k must have gotten there because it was less than p, for some i < j. 

P. V. Ramanan [SICOMP 13 (1984), 167-169] has shown how to characterize 
the permutations obtainable when m auxiliary storage locations can be used freely in 
addition to a stack. (This generalization of the problem is surprisingly difficult.) 


6. Only the trivial one, 12 ... n, by the nature of a queue. 


7 . An input-restricted deque that first outputs n must simply put the values 1, 2, 
. . . , n on the deque in order as its first, n operations. An output-restricted deque that 
first outputs n must put the values pi P 2 ■ ■ ■ p n on its deque as its first n operations. 
Therefore we find the unique answers (a) 4132, (b) 4213, (c) 4231. 

8. When n < 4, no; when n = 5, there are four (see exercise 13). 

9 . By operating backwards, we can get the reverse of the inverse of the reverse of 
any input-restricted permutation with an output-restricted deque, and conversely. This 
rule sets up a one-to-one correspondence between the two sets of permutations. 

10 . (i) There should be n X’s and n combined S’s and Q’s. (ii) The number of X’s 
must never exceed the combined number of S’s and Q’s, if we read from the left, (iii) 
Whenever the number of X’s equals the combined number of S’s and Q’s (reading from 
the left), the next character must be a Q. (iv) The two operations XQ must never be 
adjacent in this order. 

Clearly rules (i) and (ii) are necessary. The extra rules (iii) and (iv) are added to 
remove ambiguity, since S is the same as Q when the deque is empty, and since XQ 
can always be replaced by QX. Thus, any obtainable sequence corresponds to at least 
one admissible sequence. 

To show that two admissible sequences give different permutations, consider se- 
quences that are identical up to a point, and then one sequence has an S while the other 
has an X or Q. Since by (iii) the deque is not empty, clearly different permutations 
(relative to the order of the element inserted by S) are obtained by the two sequences. 
The remaining case is where sequences A, B agree up to a point and then sequence A 
has Q, sequence B has X. Sequence B may have further X’s at this point, and by (iv) 
they must be followed by an S, so again the permutations are different. 


11 . Proceeding as in exercise 4, we let g nm be the number of partial admissible 
sequences of length n, leaving m elements on the deque, not ending in the symbol X; 
hum is defined analogously, for those sequences that do end with X. We have , ,, = 

O i ? r i J\Tl~T~L)Tn 

z 9 n{m- 1 ) ~r h n (m- i)[m>lj, and /i( n + i) m = #n(m+ 1 ) + ^n(m+i )• Define G(x,z) and 
H(x, z) by analogy with the definition in exercise 4; we have 


G(x, z) = xz + 2 x 2 z 2 + 4 x 3 z 3 + (8x 4 + 2 x 2 )z 4 + (16a: 5 + 8a c 3 )z 5 + ■■■■, 
H (x, z) = z 2 + 2xz 3 + (4x 2 + 2 )z 4 + ( 8 a : 3 + 6 x) 2 5 -| . 
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G(x,z) 


xz{x — z — xh(z)) 
x — z — 2 x 2 z + xz 2 


As in exercise 4, we try to choose h(z) so that the numerator cancels with a factor of 
the denominator. We find G(x,z) = xz/(l — 22 x 2 ( 2 )) where 


r 2 (z) = 2 2 + 1 - y/(z 2 + 1) 2 - 82 2 ). 

42 

Using the convention 60 = 1, the desired generating function comes to 

|(3 - 2 - y/l - 62 + 2 2 ) = 1 + 2 + 2z 2 + 62 s + 22z 4 + 90z b + ••■. 

By differentiation we find a recurrence relation that is handy for calculation: nb n = 
3(2 n — 3)b n _i — (n — 3)b n -2, n > 2. 

Another way to solve this problem, suggested by V. Pratt, is to use context-free 
grammars for the set of strings (see Chapter 10). The infinite grammar with produc- 
tions S —¥ q n (Bx ) n , B — » sq n (Bx) n+1 B, for all n > 0, and B — * e, is unambiguous, 
and it allows us to count the number of strings with n x’s, as in exercise 2.3.4.4-31. 
12 . We have a n = 4 n /vVn 3 + 0(4 n n~ 5 ^ 2 ) by Stirling’s formula. To analyze b n , let 
us first consider the general problem of estimating the coefficient of w n in the power 
series for \/l — w \/l — aw when |a| < 1. We have, for sufficiently small a, 

y/l — w \/l — aw = Vi — w^/l — a + a(l — w) = y/l — a f ^ ^/3 fc (l — w) k+1 ^ 2 , 


where /3 = a/(l — a); hence the desired coefficient is ( — l) n y/l — cr ( 1 ^ 2 )/l fc ( k+ ^ 2 ) ■ 
Now 

n (k + l/2\ _ / n — k — 3/2 \ r(n — k — 1/2) (-1/2)^ 

V n / \ n ) T(n-|-l)r(— fc — 1/2) 

and n -fc-1/2 = £^0 [_ t !;/ / 1 2 / ! J ] 7 i"‘" 1 / 2 " , +0(n-' i - 3/2 - m ) by Eq. 1.2.11.1-(i6). Thus 
we obtain the asymptotic series [ui n ] y/l — wy/l — aw = con~ 3 ^ 2 + c\n ~ 5 ^ 2 + • • • + 
c m n -™- 3/2 + 0 (n- m - 5/2 ) where 




j + 1/2 ) a k 

k + 1/2 J (1 — 1 


For b n , we write 1 — 62 + 2 2 = (l — (3 + v/ 8 )^:) (l — (3 — \/ 8 )z) and let w = (3 + y/8 )z, 
a = (3 — \/8)/(3 + \/ 8 ), obtaining the asymptotic formula 


b 


n 


(^2-l)(3+78 ) n 

2 3 / 4 7r l/2 n 3/2 1 


+ 0(n 1 )) 


( y /2 + 1) 2 "- 1 
2 3/4 7r l/2 n 3/2 


(l + 0 (n *)). 


13. V. Pratt has found that a permutation is unobtainable if and only if it contains a 
subsequence whose relative magnitudes are respectively 


5, 2, 7, 4, ... , 4fc+l, 4fc— 2, 3, 4k, 1 


or 5, 2, 7, 4, ... , 4fc+3, 4k, 1, 4fc+2, 3 


for some k > 1 . or the same with the last two elements interchanged, or with the 1 
and 2 interchanged, or both. Thus the forbidden patterns for k = 1 are 52341, 52314, 
51342, 51324, 5274163, 5274136, 5174263, 5174236. [STOC 5 (1973), 268-277.] 
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14. (Solution by R. Melville, 1980.) Let R and S be stacks such that the queue runs 
from top to bottom of R followed by bottom to top of 5. When R is empty, pop the 
elements of S onto R until S becomes empty. To delete from the front, pop the top 
of R, which will not be empty unless the entire queue is empty. To insert at the rear, 
push onto S (unless R is empty). Each element is pushed at most twice and popped 
at most twice before leaving the queue. 

SECTION 2.2.2 

1. M - 1 ( not M). If we allowed M items, as (6) and (7) do, it would be impossible to 
distinguish an empty queue from a full one by examination of R and F, since only M 
possibilities can be detected. It is better to give up one storage cell than to make the 
program overly complicated. 

2. Delete from rear: If R = F then UNDERFLOW; Y <— X [R] : if R = 1 then R c- M. 
otherwise R <— R — 1. Insert at front: Set X[F] +- Y; if F = I then F +- M. otherwise 
F<-F-l;ifF = R then OVERFLOW. 

3. (a) LD1 I; LDA BASE, 7 : 1. This takes 5 cycles instead of 4 or 8 as in (8). 

(b) Solution 1: LDA BASE, 2:7 where each base address is stored with Ii = 0, 
I2 = 1- Solution 2: If it is desired to store the base addresses with Id = I 2 = 0, we 
could write LDA X2,7: 1 where location X2 contains NOP BASE, 2: 7. The second solution 
takes one more cycle, but it allows the base table to be used with any index registers. 

(c) This is equivalent to “LD4 X(0 : 2)”, and takes the same execution time, except 
that rI4 will be set to +0 when X(0:2) contains -0. 

4. (a) NOP *,7. (b) LDA X, 7: 7(0: 2). (c) This is impossible; the code LDA Y,7:7 
where location Y contains NOP X,7:7 breaks the restriction on 7:7. (See exercise 5.) 

(d) LDA X,7: 1 with the auxiliary constants 

X NOP *+1,7:2 
NOP *+1,7:3 
NOP *+1,7:4 
NOP 0,5:6 

The execution time is 6 units, (e) INC6 X, 7:6 where X contains NOP 0,6:6. 

5. (a) Consider the instruction ENTA 1000,7 :7 with the memory configuration 


location ADDRESS 

Ii 

I 2 

1000 

1001 

7 

7 

1001 

1004 

7 

1 

1002 

1002 

2 

2 

1003 

1001 

1 

1 

1004 

1005 

1 

7 

1005 

1006 

1 

7 

1006 

1008 

7 

7 

1007 

1002 

7 

1 

1008 

1003 

7 

2 


and with rll = 1, rI2 = 2. We find that 1000,7,7 = 1001,7,7,7 = 1004,7,1.7.7 = 

1005.1.7.1.7.7 = 1006,7,1,7,7 = 1008,7,7,1,7,7 = 1003,7,2,7,1,7,7 = 1001.1,1,2,7,1,7,7 
= 1002,1,2,7,1,7,7 = 1003,2,7,1,7,7 = 1005,7,1,7,7 = 1006,1,7,1,7,7 = 1007,7,1,7.7 = 

1002.7.1.1.7.7 = 1002,2,2,1,1,7,7 = 1004,2,1,1,7,7 = 1006,1,1,7,7 = 1007,1,7,7 = 1008,7,7 
= 1003,7,2,7 = 1001,1,1,2,7 = 1002,1,2,7 = 1003,2,7 = 1005,7 = 1006,1,7 = 1007,7 = 
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1002,7,1 = 1002,2,2,1 = 1004,2,1 = 1006,1 = 1007. (A faster way to do this derivation 
by hand would be to evaluate successively the addresses specified in locations 1002, 
1003, 100 1 , 1008, 1005, 1006, 1004, 1001, 1000, in this order; but a computer evidently 
needs to go about the evaluation essentially as shown.) The author tried out several 
fancy schemes for changing the contents of memory while evaluating the address, with 
everything to be restored again by the time the final address has been obtained. Similar 
algorithms appear in Section 2.3.5. However, these attempts were unfruitful and it 
appears that there is just not enough room to store the necessary information. 

(b,c) Let H and C be auxiliary registers and let N be a counter. To get the effective 
address M, for the instruction in location L, do the following: 

Al. [Initialize.] Set H <— 0, C <— L, N <— 0. (In this algorithm, C is the “current” 
location, H is used to add together the contents of various index registers, and 
N measures the depth of indirect addressing.) 

A2. [Examine address.] Set M <- ADDRESS (C). If Ii(C) = J, 1 < j < 6, set 
M •(— M + rlj. If I 2 (C) = j, 1 < j < 6, set H <— H + rlj. If i! (C) = I 2 (C) = 7, 
set N N + 1, H <— 0. 

A3. [Indirect?] If either Ir CC) or I 2 (C) equals 7, set C ■<— M and go to A2. 
Otherwise set H <- H + H, H f- 0. 

A4. [Reduce depth.] If N > 0, set C <— M, N ■<— N — 1, and go to A2. Otherwise M is 
the desired answer. | 

This algorithm will handle any situation correctly except those in which Ii = 7 
and 1 < I 2 < 6 and the evaluation of the address in ADDRESS involves a case with 
■*■1 = 1 2 = 7. The effect is as if I 2 were zero. To understand the operation of Algo- 
rithm A, consider the notation of part (a); the state “L, 7, 1,2, 5, 2, 7, 7, 7, 7” is represented 
by C or H = L, N = 4 (the number of trailing 7s), and H = rll + rI2 + rI5 + rI2 (the 
post-indexing). In a solution to part (b) of this exercise, the counter N will always be 
either 0 or 1. 

6. (c) causes OVERFLOW, (e) causes UNDERFLOW, and if the program resumes it causes 
OVERFLOW on the final I 2 . 

7. No, since T0P[i] must be greater than 0LDT0P [i] . 

8. With a stack, the useful information appears at one end with the vacant informa- 
tion at the other: 

A B C 


where A — BASE[j] , B — TOP [j] , C = BASE[j + 1] . With a queue or deque, the useful 
information appears at the ends with the vacant information somewhere in the middle: 

A B C D 


or in the middle with the vacant information at the ends: 


AC B D 



where A = BASElj] , B = REAR[j] , C = FRONT [j] , D = BASE [j + 1] . The two cases 
are distinguished by the conditions B < C and B > C , respectively, in a nonempty 
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queue; or. if the queue is known not to have overflowed, the distinguishing conditions 
are respectively B < C and B > C. The algorithms should therefore be modified in an 
obvious way so as to widen or narrow the gaps of vacant information. (Thus in case 
of overflow, when B = C, we make empty space between B and C by moving one part 
and not the other.) In the calculation of SUM and D[)] in step G2, each queue should 
be considered to occupy one more cell than it really does (see exercise 1). 

9. Given any sequence specification ai,a2,...,a m there is one move operation re- 
quired for every pair (j,k) such that j < k and a, > a k . (Such a pair is called an 
“inversion”; see Section 5.1.1.) The number of such pairs is therefore the number of 
moves required. Now imagine all n m specifications written out, and for each of the 
(” ! ) pairs (j, k) with j < k count how many specifications have a j > a k . Clearly this 
equals (£) , the number of choices for a 3 and a k , times n m ~ 2 , the number of ways to fill 
in the remaining places. Hence the total number of moves among all specifications is 
(’2 ) ("Jfi” “• Divide this by n m to get the average, Eq. (14). 

10 . As in exercise 9 we find that the expected value is 


(7) E Pip* 

1 < j < k < n 


2(2 )^ Pl + ~ (Pl + f Pn)) 

^(7)( 1 - (Pi + •• • +Pn ))■ 


For this model, it makes absolutely no difference what the relative order of the lists 
is! (A moment’s reflection explains why; if we consider all possible permutations of a 
given sequence ai,...,a m , we find that the total number of moves summed over all 
these permutations depends only on the number of pairs of distinct elements aj / a. k .) 

11. Counting as before, we find that the expected number is 


E mn t — 


££(*-*) 


k -2 

r 


(n-l) k 


here r is the number of entries in an, a 2 , a k ~ 1 that equal a k . This quantity can 
also be expressed in the simpler form 


Emnt — 


E 

k>t 


(n-l)” 


(T)). 


for t > 0. 


Is there a simpler way yet to give the answer? Apparently not, since the generating 
function for given n and t is 


J 2 e 


mnt Z — 


n — 1 


z ( z \*+i 

- (r ~_ 1)J (* + (!-*)«(* + !)). 


2 n (1 — z ) 3 \ n — (n — 1)^ 

12. If m, = 2 k, the average is 2 ~ 2k times 

O + (f)( 2k - d + • • ■ + (7 ) fc + ( k + J (fc +!)+•••+ 


The latter sum is 


/ 2 k 
V k 
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A similar argument may be used when m = 2 A + 1. The answer is 


m m i rn — 1 \ 
~2~ + 2™ V [m/2] ) ' 


13 . A. C. Yao has proved that we have Emax(Ai, k 2 ) = \m + (27t(1 - 2 p)) 1/2 N /m + 
0(m _1/2 (logm) 2 ) for large m, when p < [SICOMP 10 (1981), 398-403.] And 
P. Flajolet has extended the analysis, showing in particular that the expected value is 
asymptotically am when p = | , where 


= +8 E 


sin(n7r/2) cosh(n7r/2) 
n 2 7r 2 sinh nn 


0.6753144833. 


Moreover, when p > | the final value of Ai tends to be uniformly distributed as m — > oo, 
so Emax(Ai,Aa) « § m. [See Lecture Notes in Comp. Sci. 233 (1986), 325-340.] 

14. Let kj = m/n + i/ mxj . (This idea was suggested by N. G. de Bruijn.) Stirling’s 
approximation implies that 


fcr ! . . . fen ! 


maxffci, . . . , k n ) 


(V2 jr) 1 n n n ^~ ^ — + \fm maxfii, . . . , £ n )j 

X 6XP (“f + • • • + x l)) is/m.) 1 


~ m ( 1 -of ' 


when k\ + • • • + k„ = m and when the x’s are uniformly bounded. The sum of the latter 
quantity over all nonnegative Ai, . . . , k n satisfying this condition is an approximation 
to a Riemann integral; we may deduce that the asymptotic behavior of the sum is 
a n (m/n) + c n i/m + 0(1), where 

a n — (V2n) n n n/2 f exp(-^(aqH f x 2 n )\ dx 2 . . . dx n , 

■>Xli |-a- n =0 V ' 


c n = (V 2 tt) 1 n ri n/2 [ max(ii, . . ,,x n ) exp — (k 2 + hx 2 )) dx 2 . . ■ dx n , 

J Xl + ---+x n =0 \ Z / 

since it is possible to show that the corresponding sums come within e of a n and c„ for 
any e. 

We know that a„ = 1, since the corresponding sum can be evaluated explicitly. 
The integral that appears in the expression for c„ equals nl\, where 

h = f xi+ , +Xn=0 nexp (~^(®? + bz 2 )) dx 2 ...dx n . 

Xl >X2,...,X n 

We may make the substitution 


*1 — —(3/2 H \~y n ), x 2 = X! - y 2 , x 3 = X! - y 3 , ..., x n = x\ - y„\ 

then we find h = h/n 2 , where 


/ ( 2/2 + dy 2 ...dy n , 

J y2, --,yn>o ' ^ ’ 
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and Q — n(y 2 4 h J/S ) — (j/2 H b y n ) 2 ■ Now by symmetry, I 2 is ( n - 1) times the 

same integral with (y 2 + b y„) replaced by y 2 \ hence I 2 = (n — l)/ 3 , where 

J 3= (ny 2 — (y 2 H b y n )) expf-^) dy 2 . . . dy n 

J y2,- -,yn>0 \ ^ / 

= / eX P 

J y3,---,yn>o ' ^ ' 

here Q 0 is Q with y 2 replaced by zero. [When n = 2, let I 3 = 1.] Now let z 3 = 

\Znyj — (j/3 4 by n )/(\/2 + \/n ), 3 < j < n. Then Q 0 = z 2 + ■ ■ ■ + z „ , and we deduce 

that I 3 = /4/n* n-3 ^ 2 \/2, where 

h= [ exp - + '-' + Z " ) 

^ y3T-,y n >o V ^ / 

= ac n J exp " 3 + - + — j dz 3 ...dz n = a n (V 2^) n ~ 2 , 

where a n is the “solid angle” in (n — 2)-dimensional space spanned by the vectors 
(n + \/2n, 0 , . . . , 0 ) — ( 1 , 1 , . . . , 1 ), . . . , ( 0 , 0 , ... , n + \/2n ) — ( 1 , 1 , ... , 1), divided by 
the total solid angle of the whole space. Hence 


Cn — 



We have a 2 = 1, a 9 = a 4 — tt 1 arctan y/2 « .304, and 


13 1 

as = - + — ■ arctan — = « .206. 

8 4tt ^8 

[The value of c 3 was found by Robert M. Kozelka, Annals of Math. Stat. 27 (1956), 
507-512, but the solution to this problem for higher values of n has apparently never 
appeared in the literature.] 

16. Not unless the queues meet the restrictions that apply to the primitive method of 
(4) and (5). 

17. First show that BASE [j] 0 < BASE [j] x at all times. Then observe that each overflow 
for stack i in So(o ) that does not also overflow in si(<r) occurs at a time when stack i 
has gotten larger than ever before, yet its new size is not less than the original size 
allocated to stack i in si(cr). 

18. Suppose the cost of an insertion is a, plus bN + cn if repacking is needed, where 
N is the number of occupied cells; let the deletion cost be d. After a repacking that 
leaves N cells occupied and S = M — N cells vacant, imagine that each insertion until 
the next repacking costs a + b + 10c + 10(5 + c)nN/S = 0(1 + na/( 1 - a)), where 
a = N/M. If p insertions and q deletions occur before that repacking, the imagined 
cost is p(a + b+ 10c + 10(6 + c)nN / S) + qd, while the actual cost is pa + bN 1 +cn + qd < 
pa + pb + bN + cn + qd. The latter is less than the imagined cost, because p > .1 S/n; 
our assumption that M > n 2 implies that cS/n + (6 + c)N > bN + cn. 

19. We could simply decrease all the subscripts by 1; the following solution is slightly 
nicer. Initially T = F = R = 0. 

Push Y onto stack X: If T = M then OVERFLOW; X[T] <— Y; T 4 - T+ 1. 

Pop Y from stack X: If T = 0 then UNDERFLOW; T4-T-1;Y*-X[T]. 
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Insert Y into queue X: X[R] 4- Y; R 4- (R + 1) mod M; if R = F then OVERFLOW. 
Delete Y from queue X: if F = R then UNDERFLOW; Y f- X [F] ; F 4 — (F + 1) mod M. 

As before, T is the number of elements on the stack, and (R — F) modM is the number 
of elements on the queue. But the top stack element is now X[T - 1], not X[T]. 

Even though it is almost always better for computer scientists to start counting 
at 0, the rest of the world will probably never change to 0-origin indexing. Even Edsger 
Dijkstra counts “1-2-3-4 | 1-2-3-4” when he plays the piano! 


SECTION 2.2.3 

1. OVERFLOW is implicit in the operation P 4= AVAIL. 


INSERT 

STJ 

IF 

Store location of “NOP T” . 


STJ 

9F 

Store exit location. 


LD1 

AVAIL 

rll 4= AVAIL. 


J1Z 

OVERFLOW 



LD3 

0,1 (LINK) 



ST3 

AVAIL 



STA 

0,1 (INFO) 

INFO(rll) 4- Y. 

1H 

LD3 

*(0:2) 

rI3 4- LOC(T) . 


LD2 

0,3 

rI2 4 - T. 


ST2 

0,1 (LINK) 

LINK(rll) 4- T. 


ST1 

0,3 

T 4- rll. 

9H 

JMP 

* 

1 

DELETE 

STJ 

IF 

Store location of “NOP T” . 


STJ 

9F 

Store exit location. 

1H 

LD2 

*(0:2) 

rI2 4- L0C(T) . 


LD3 

0,2 

rI3 4- T. 


J3Z 

9F 

Is T = A? 


LD1 

0,3(LINK) 

rll 4- LINK(T) . 


ST1 

0,2 

T 4- rll. 


LDA 

0,3(INF0) 

rA 4- INFO (rll). 


LD2 

AVAIL 

AVAIL 4= rI3. 


ST2 

0,3 (LINK) 



ST3 

AVAIL 



ENT3 

2 

Prepare for second exit. 

9H 

JMP 

* ,3 

1 

OVERFLOW STJ 

9F 

Store setting of rj. 


ST1 

8F(0 : 2) 

Save rll setting. 


LD1 

P00LMAX 



ST1 

AVAIL 

Set AVAIL to new location. 


INC1 c 



ST1 

POOLMAX 

Increment POOLMAX. 


CMP1 SEQMIN 



JG 

TOOBAD 

Has storage been exceeded? 


STZ 

i -c, l(LINK) Set LINK (AVAIL) 4— A. 

9H 

ENT1 * 

Take rj setting. 


DEC1 2 

Subtract 2. 


ST1 

*+2(0:2) 

Store exit location. 

8H 

ENT1 * 

Restore rll. 


JMP 

* 

Return. | 
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5. Inserting at the front is essentially like the basic insertion operation (8), with an 
additional test for empty queue: P <= AVAIL, INFO(P) <— Y, LINK(P) F; if F = A then 
R <— P: F <— P. 

To delete from the rear, we would have to find which node links to NODE(R), and 
that is necessarily inefficient since we have to search all the way from F. This could be 
done, for example, as follows: 

a) If F = A then UNDERFLOW, otherwise set P <— LOC(F) . 

b) If LINK(P) / R then set P •(— LINK(P) and repeat this step until LINK(P) = R. 

c) Set Y INFO(R) , AVAIL <= R. R -f- P, LINK(P) <- A. 

6 . We could remove the operation LINK(P) A from (14), if we delete the commands 
“F «— LINK(P)' and '‘if F = A then set R •(— LOC(F)” from (17); the latter are to be 
replaced by “if F = R then F «— A and R <— LOC(F), otherwise set F •<— LINK(P)”. 

The effect of these changes is that the LINK field of the rear node in the queue will 
contain spurious information that is never interrogated by the program. A trick like 
this saves execution time and it is quite useful in practice, although it violates one of 
the basic assumptions of garbage collection (see Section 2.3.5) so it cannot be used in 
conjunction with such algorithms. 

7 . (Make sure that your solution works for empty lists.) 

11. Set P <r- FIRST, q <— A. 

12 . If P 7^ A, set R «— Q, Q P, P ■<— LINK(Q) , LINK(Q) t— R, and repeat this step. 

13 . Set FIRST <- Q. | 

In essence we are popping nodes off one stack and pushing them onto another. 


LD1 

FIRST 

1 

IL P = rll <- FIRST. 

ENT2 

0 

1 

Q = rI2 <- A. 

J1Z 

2F 

1 

12. If the list is empty, jump 

ENTA 

0,2 

n 

R = r A 7 — Q . 

ENT2 

0,1 

n 

q <- p. 

LD1 

0,2(LINK) 

n 

P «- LINK(Q). 

STA 

0,2(LINK) 

n 

LINK(Q) 4- R. 

J1NZ 

IB 

n 

Is P ^ A? 

ST2 

FIRST 

1 

IS. FIRST <- Q. | 


The time is (7 n + 6)u. Better speed (5 n + constant^ is attainable; see exercise 1.1 -3. 

9. (a) Yes. (b) Yes, if biological parenthood is considered; no, if legal parenthood 
is considered (a man’s daughter might marry his father, as in the song “I’m My Own 
Grampa”). (c) No ( — 1 -< 1 and 1 X — 1). (d) Let us hope so, or else there is a circular 
argument. (e) 1 -< 3 and 3^1. (f) The statement is ambiguous. If we take the 

position that the subroutines called by y are dependent upon which subroutine calls y, 
we would have to conclude that the transitive law does not necessarily hold. (For 
example, a general input-output subroutine might call on different processing routines 
for each I/O device present, but these processing subroutines are usually not all needed 
in a single program. This is a problem that plagues many automatic programming 
systems.) 

10. For (i) there are three cases: x — y; x C y and y = z; x C y and y C z. For (ii) 
there are two cases: x = y\ x ^ y. Each case is handled trivially, as is (iii). 

11 . “Multiply out” the following to get all 52 solutions: 13749(25 + 52)86 + (1379 + 
1397 + 1937 + 9137)(4258 + 4528 + 2458 + 5428 + 2548 + 5248 + 2584 + 5284)6 + (1392 + 
1932 + 1923 + 9123 + 9132 + 9213)7(458 + 548 + 584)6. 
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12 . For example: (a) List all sets with k elements (in any order) before all sets with 
k + 1 elements, 0 < k < n. (b) Represent a subset by a sequence of Os and Is showing 
which elements are in the set. This gives a correspondence between all subsets and the 
integers 0 through 2” — 1, via the binary number system. The order of correspondence 
is a topological sequence. 

13 . Sha and Kleitman, Discrete Math. 63 (1987), 271-278, have proved that the 

number is at most []fc = o (£)^- This exceeds the obvious lower bound nLo ('()■ = 

2 2 b‘+o(i° g n)) ky a factor 0 f e 2 +o(n). ^ey conjecture that the lower bound is closer 
to the truth. 

14 . If oi a 2 . . . a n and bi b 2 . . . b n are two possible topological sorts, let j be minimal 
such that a,j / bj\ then a*, = bj and cij = b m for some k,m > j. Now bj aj since 
k > ji and aj bj since m > j, hence (iv) fails. Conversely if there is only one 
topological sort ai a 2 . . . a n , we must have aj X a 7 + i for 1 < j < n, since otherwise aj 
and ajj - 1 could be interchanged. This and transitivity imply (iv). 

Note: The following alternative proofs work also for infinite sets, (a) Every partial 
ordering can be embedded in a linear ordering. For if we have two elements with x 0 ± y 0 
and y 0 x 0 we can generate another partial ordering by the rule u x A y or ( x -< xo and 
Vo r< y ). ’’The latter ordering “includes” the former and has x 0 < yo- Now apply Zorn’s 
lemma or transfinite induction in the usual way to complete the proof, (b) Obviously 
a linear ordering cannot be embedded in any different linear ordering, (c) A partial 
ordering that has incomparable elements xo and yo as in (a) can be extended to two 
linear orderings in which xq A y 0 and y 0 A x 0 , respectively, so at least two linear 
orderings exist. 

15 . If S is finite, we can list all relations a < b that are true in the given partial 
ordering. By successively removing, one at a time, any relations that are implied by 
others, we arrive at an irredundant set. The problem is to show there is just one such 
set, no matter in what order we go about removing redundant relations. If there were 
two irredundant sets U and V, in which “a -< b” appears in U but not in V, there are 
k + 1 relations a -< ci -< ■ • • -< Ck -< b in V for some k > 1. But it is possible to deduce 
a < c i and ci -< 6 from U, without using the relation a -< b (since b ^ ci and c\ -£ a), 
hence the relation a -< b is redundant in U . 

The result is false for infinite sets S , when there is at most one irredundant set 
of relations. For example if S denotes the integers plus the element oo and we define 
n -< n + 1 and n ^ oo for all n, there is no irredundant set of relations that characterizes 
this partial ordering. 

16 . Let x Pl x P2 ... x Pn be a topological sorting of S ; apply the permutation prp 2 ...p n 
to both rows and columns. 

17 . If k increases from 1 to n in step T4, the output is 1932745860. If k decreases 
from n to 1 in step T4, as it does in Program T, the output is 9123745860. 

18 . They link together the items in sorted order: QLINK [0] first, QLINK [QLINK [0] ] 
second, and so on; QLINK [last] = 0. 

19 . This would fail in certain cases; when the queue contains only one element in 
step T5, the modified method would set F = 0 (thereby emptying the queue), but other 
entries could be placed in the queue in step T6. The suggested modification would 
therefore require an additional test of F = 0 in step T6. 

20 . Indeed, a stack could be used, in the following way. (Step T7 disappears.) 
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T4. Set T <— 0. For 1 < A < n if COUNT [A] is zero do the following: Set 
SLINK [A] 4- T, T 4- A. (SLINK [A] = QLINK [A] .) 

T5. Output the value of T. If T = 0, go to T8; otherwise, set N 4- N — 1, P 4- T0P[T] . 
T 4- SLINK [T] . 

T6. Same as before, except go to T5 instead of T7; and when COUNT [SUC CP) ] goes 
down to zero, set SLINK [SUC (P)] 4- T and T 4- SUC(P). 

21. Repeated relations only make the algorithm a little slower and take up more space 
in the storage pool. A relation "j -< j y would be treated like a loop (an arrow from a 
box to itself in the corresponding diagram), which violates partial order. 

22. To make the program “fail-safe” we should (a) check that 0 < n < some appropri- 
ate maximum; (b) check each relation j <k for the conditions 0 < j, k < n; (c) make 
sure that the number of relations doesn’t overflow the storage pool area. 

23. At the end of step T5, add “T0P[F] 4- A”. (Then at all times T0P[1] TOP [n] 

point to all the relations not yet canceled.) In step T8, if N > 0, print “LOOP DETECTED 
IN INPUT:”, and set QLINK [A] 4- 0 for 1 < k < n. Now add the following steps: 

T9. For 1 < A < n set P 4- TOP [A:] , TOP [A:] 4- 0, and perform step T10. (This 
will set QLINK [j] to one of the predecessors of object j, for each j not yet 
output.) Then go to Til. 

T10. If P / A, set QLINK [SUC (P)] <— A, P 4- NEXT(P), and repeat this step. 

Til. Find a k with QLINK [A] ^ 0. 

T12. Set TOP [A:] 4- 1 and k 4- QLINK [A] . Now if TOP [A’] = 0, repeat this step. 
T13. (We have found the start of a loop.) Print the value of k, set TOP [A] 4- 0. 
k 4— QLINK [A - ] , and if TOP [A] = 1 repeat this step. 

T14. Print the value of A (the beginning and end of the loop) and stop. (Note: 
The loop has been printed backwards; if it is desired to print the loop in 
forward order, an algorithm like that in exercise 7 should be used between 
steps T12 and T13.) | 

24. Insert three lines in the program of the text: 

08a PRINTER EQU 18 

14a ST6 NO 

59a STZ X, 1 (TOP) TOP [F] 4- A. 

Replace lines 74-75 by the following: 


74 

J6Z 

DONE 


75 

OUT 

LINE1 (PRINTER) 

Print indication of loop. 

76 

LD6 

NO 


77 

STZ 

X, 6 (QLINK) 

QLINK [A] 4- 0. 

78 

DEC6 

1 


79 

J6P 

*-2 

n > A > 1. 

80 

LD6 

NO 


81 T9 

LD2 

X,6(T0P) 

P 4- TOP [A] . 

82 

STZ 

X,6(T0P) 

TOP [A] 4- 0. 

83 

J2Z 

T9A 

Is P = A? 

84 T10 

LD1 

0,2(SUC) 

rll 4- SUC(P). 

85 

ST6 

X,1 (QLINK) 

QLINK [rll] 4- A. 

86 

LD2 

0,2 (NEXT) 

P 4- NEXT(P) . 
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87 


J2P 

T10 

Is P # A? 

88 

T9A 

DEC6 

1 


89 


J6P 

T9 

n > k > 1. 

90 

Til 

INC6 

1 


91 


LDA 

X,6(QLINK) 


92 


JAZ 

*-2 

Find k with QLINKfft] / 

93 

T12 

ST6 

X,6(T0P) 

TOP [ft] 4- k. 

94 


LD6 

X,6(QLINK) 

k 4- QLINK Ik] . 

95 


LD1 

X,6(T0P) 


96 


J1Z 

T12 

Is TOP [ft] = 0? 

97 

T13 

ENTA 

0,6 


98 


CHAR 


Convert k to alphameric. 

99 


JBUS 

* (PRINTER) 


100 


STX 

VALUE 

Print. 

101 


OUT 

LINE2 (PRINTER) 


102 


J1Z 

DONE 

Stop when TOP [ft] = 0. 

103 


STZ 

X,6(T0P) 

TOP [ft] 4- 0. 

104 


LD6 

X,6(QLINK) 

k 4- QLINK [ft]. 

105 


LD1 

X,6(T0P) 


106 


JMP 

T13 


107 

LINE1 

ALF 

LOOP 

Title line 

108 


ALF 

DETEC 


109 


ALF 

TED I 


110 


ALF 

N INP 


111 


ALF 

UT: 


112 

LINE2 

ALF 


Succeeding lines 

113 

VALUE 

EQU 

LINE2+3 


114 


0RIG 

LINE2+24 


115 

DONE 

HLT 


End of computation 

116 

X 

END 

T0PS0RT 

1 


Note: If the relations 9 -< 1 and 6 — < 9 are added to the data (18), this program 
will print “9, 6, 8, 5, 9” as the loop. 

26. One solution is to proceed in two phases as follows: 

Phase 1. (We use the X table as a (sequential) stack as we mark B = 1 or 2 for each 
subroutine that needs to be used.) 

AO. For 1 < J < N set B(X[J]) 4- B(X[J]) +2, if B(X[J] ) < 0. 

Al. If N = 0, go to phase 2; otherwise set P 4— X[N] and decrease N by 1. 

A2. If |B (P ) | = 1, go to Al, otherwise set P 4— P + 1. 

A3. If B(SUB1 (P) ) < 0, set N 4- N + 1, B(SUB1(P)) 4- B(SUB1(P)) + 2, X[N] 4- 
SUB1(P). If SUB2(P) / 0 and B(SUB2(P)) < 0, do a similar set of actions with 
SUB2CP). Go to A2. | 

Phase 2. (We go through the table and allocate memory.) 

Bl. Set P 4- FIRST. 

B2. If P = A, set N 4— N + 1 , BASE(L0C(X [N] ) ) 4- MLOC, SUB(L0C(X [N] ) ) 4- 0, and 
terminate the algorithm. 
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B3. If B(P) > 0, set N «— N + 1, BASE(L0C(X [N] ) ) MLOC, SUB(L0C(X [N] ) ) +- P. 
MLOC 4- MLOC + SPACE (P) . 

B4. Set P 4— LINK(P) and return to B2. | 

27. Comments on the following code are left to the reader. 


B 

EQU 

0:1 

A1 

J1Z 

B1 


INC1 

1 

SPACE 

EQU 

2:3 


LD2 

X.l 


INCA 

2 

LINK 

EQU 

4:5 


DEC1 

1 


STA 

0.3(B) 

SUB1 

EQU 

2:3 

A2 

LDA 

0, 2(1:1) 


ST3 

X,1 

SUB2 

EQU 

4:5 


DECA 

1 


JMP 

A2 

BASE 

EQU 

0:3 


JAZ 

A1 

B1 

ENT2 

FIRST 

SUB 

EQU 

4:5 


INC2 

1 


LDA 

MLOC 

AO 

LD2 

N 

A3 

LD3 

0,2(SUB1) 


JMP 

IF 


J2Z 

B1 


LDA 

0.3(B) 

B3 

LDX 

0.2(B) 

1H 

LD3 

X , 2 


JAP 

9F 


JXNP 

B4 


LDA 

0,3(B) 


INC1 

1 


INC1 

1 


JAP 

*+3 


INCA 

2 


ST2 

X , 1 (SUB) 


INCA 

2 


STA 

0.3(B) 


ADD 

0,2 (SPACE) 


STA 

0.3(B) 


ST3 

X.l 

1H 

STA 

X+l , 1 (BASE) 


DEC 2 

1 

9H 

LD3 

0,2(SUB2) 

B4 

LD2 

0,2(LINK) 


J2P 

IB 


J3Z 

A2 

B2 

J2NZ 

B3 


LD1 

N 


LDA 

0.3(B) 


STZ 

X+l, 1 (SUB) 





JAP 

A2 





28. We give here only a few comments related to the military game. Let A be the 
player with three men whose pieces start on nodes A13; let B be the other player. In 
this game, A must “trap' B , and if B can cause a position to be repeated for a second 
time we can consider B the winner. To avoid keeping the entire past history of the 
game as an integral part of the positions, however, we should modify the algorithm in 
the following way: Start by marking the positions 157-4, 789-B, 359-6 with B to move 
as “lost” and apply the suggested algorithm. Now the idea is for player A to move 
only to B's lost positions. But A must also take precautions against repeating prior 
moves. A good computer game-playing program will use a random number generator 
to select between several winning moves when more than one is present, so an obvious 
technique would be to make the computer, playing A, just choose randomly among 
those moves that lead to a lost position for B. But there are interesting situations 
that make this plausible procedure fail! For example, 
consider position 258-7 with A to move; this is a 
won position. From position 258-7. player A might 
try moving to 158-7 (which is a lost position for B , 
according to the algorithm). But then B plays to 
158-B, and this forces A to play to 258-B, after which 
B plays back to 258-7; B has won, since the former 
position has been repeated! This example shows that 
the algorithm must be re-invoked after every move 
has been made, starting with each position that has 
previously occurred marked “lost” (if A is to move) or 
“won” (if B is to move). The military game makes a Board for the “military game.” 
very satisfactory computer demonstration program. 
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29. (a) If FIRST = A, do nothing; otherwise set P 4— FIRST, and then repeatedly set 

P 4— LINK(P) zero or more times until LINK(P) = A. Finally set LINK(P) 4— AVAIL and 
AVAIL 4 — FIRST (and probably also FIRST 4— A), (b) If F = A, do nothing; otherwise 

set LINK(R) 4- AVAIL and AVAIL 4 - F (and probably also F 4— A, R 4— LOC(F)). 

30. To insert, set P 4= AVAIL, INFO(P) 4 - Y, LINK(P) 4 - A, if F = A then F 4- P 
else LINK(R) 4— P, and R 4— P. To delete, do ( 9 ) with F replacing T. (Although it 
is convenient to let R be undefined for an empty queue, this lack of discipline might 
confuse a garbage collection algorithm, as in exercise 6 .) 


SECTION 2.2.4 

1 . No, it does not help; it seems to hinder, if anything. (The stated convention is not 
especially consistent with the circular list philosophy, unless we put N0DE(L0C(PTR) ) 
into the list as its list head.) 



3. If PTRi = PTR 2 , the only effect is PTR 2 4— A. If PTRi / PTR 2 , the exchange of links 
breaks the list into two parts, as if a circle had been broken in two by cutting at two 
points; the second part of the operation then makes PTRi point to a circular list that 
consists of the nodes that would have been traversed if, in the original list, we followed 
the links from PTRi to PTR 2 . 


4. Let HEAD be the address of the list head. To push down Y onto the stack: Set 
P 4= AVAIL. INFO (P) 4 - Y, LINK(P) 4- LINK (HEAD), LINK (HEAD) 4- P. To pop up the 
stack onto Y: If LINK (HEAD) = HEAD then UNDERFLOW; otherwise set P 4- LINK (HEAD), 
LINK (HEAD) 4- LINK(P), Y 4- INFO(P), AVAIL 4= P. 

5. (Compare with exercise 2. 2. 3-7.) Set Q 4— A, P 4 — PTR. and then while P ^ A 
repeatedly set R 4 - Q. Q 4 — P, P 4 — LINK(Q) , LINK(Q) 4- R. (Afterwards Q = PTR.) 


6 . 


(a) 

(b) 



7. Matching terms in the polynomial are located in one pass over the list, so repeated 
random searches are avoided. Also, increasing order would be incompatible with the 
1 ” sentinel. 


8. We must know what node points to the current node of interest, if we are going to 
delete that node or to insert another one ahead of it. There are alternatives, however: 
We could delete NODE(Q) by setting Q2 4— LINK(Q) and then setting NODE(Q) 4— 
N0DE(Q2), AVAIL 4= Q2; we could insert a N0DE(Q2) in front of NODE(Q) by first 
interchanging N0DE(Q2) 4-4 NODE(Q), then setting LINK(Q) 4 — Q2. Q 4— Q2. These 
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clever tricks allow the deletion and insertion without knowing which node links to 
NODE(Q); they were used in early versions of IPL. But they have the disadvantage that 
the sentinel node at the end of a polynomial will occasionally move, and other link 
variables may be pointing to this node. 

9 . Algorithm A with P = Q simply doubles polynomial ( C) ) , as it should — except 
in the anomalous case that COEF = 0 for some term with ABC > 0. when it fails 
badly. Algorithm M with P = M also gives the expected result. Algorithm M with 
P = Q sets polynomial (P) <- polynomial (P) times (1 + t x )(l + + t k ) if 

M = ti + t 2 + ■ ■ ■ + t k (although this is not immediately obvious). When M = Q. 
Algorithm M surprisingly gives the expected result, polynomial (Q) polynomial (Q) 
+ polynomial (Q) x polynomial (P), except that the computation blows up when the 
constant term of polynomial (P) is —1. 

10 . None. (The only possible difference would be in step M2, removing error checks 
that A, B, or C might individually overflow; these error checks were not specified because 
we assumed that they were not necessary.) In other words, the algorithms in this section 
may be regarded as operations on the polynomial f(x b2 ,x b ,x) instead of on f(x,y,z). 


11. COPY 

STJ 

9F 

(comments 

ST6 

1,3 (LINK) 

If; 

ENT3 

9F 

are 

ENT3 

0,6 


LDA 

1,1 

left 

LD1 

1,1 (LINK) 

1H 

LD6 

AVAIL 

to 

LDA 

1,1 


J6Z 

OVERFLOW 

the 

JANN 

IB 


LDX 

1 ,6(LINK) 

reader) 

LD2 

8F (LINK) 


STX 

AVAIL 


ST2 

1,3 (LINK) 


STA 

1,6 


9H JMP 

* 


LDA 

0,1 


8H CON 

0 1 


STA 

0,6 





12. Let the polynomial copied have p terms. Program A takes (29p + 13)w, and to 
make it a fair comparison we should add the time to create a zero polynomial, say 18u 
with exercise 14. The program of exercise 11 takes (2 lp + 31)u, about § as much. 


13 . 

ERASE STJ 

9F 




LDX 

AVAIL 




LDA 

1,1 (LINK) 




STA 

AVAIL 




STX 

1,1 (LINK) 



9H 

JMP 

* 1 


14 . 

ZERO 

STJ 

9F 

MOVE IF (2) 



LD1 

AVAIL 

ST2 1,2 (LINK) 



J1Z 

OVERFLOW 

9H JMP * 



LDX 

1,1 (LINK) 

1H CON 0 



STX 

AVAIL 

CON -1 (ABC) 



ENT 2 

0,1 


15 . 

MULT 

STJ 

9F 

Entrance to subroutine 



LDA 

5F 

Change settings of switches 



STA 

SW1 




LDA 

6F 




STA 

SW2 




STA 

SW3 



2-2.5 ANSWERS TO EXERCISES 553 



JMP 

*42 


2H 

JMP 

ADD 

M2. Multiply cycle. 

1H 

LD4 

1 ,4(LINK) 

Ml. Next multiplier. M 4— LINK(M) 


LDA 

1,4 



JANN 

2B 

To M2 if ABC(M) > 0. 

8H 

LDA 

7F 

Restore settings of switches. 


STA 

SW1 



LDA 

8F 



STA 

SW2 



STA 

SW3 


9H 

JMP 

* 

Return. 

5H 

JMP 

*41 

New setting of SW1 


LDA 

0,1 



MUL 

0,4 

rX 4- C0EF(P) x C0EF(M). 


LDA 

1,1 (ABC) 

ABC(P) 


JAN 

*42 



ADD 

1,4 (ABC) 

+ ABC(M) , if ABC(P) > 0. 


SLA 

2 

Move into 0:3 field of rA. 


STX 

OF 

Save rX for use in SW2 and SW3. 


JMP 

SW141 


6H 

LDA 

OF 

New setting of SW2 and SW3 

7H 

LDA 

1,1 

Usual setting of SW1 

8H 

LDA 

0,1 

Usual setting of SW2 and SW3 

OH 

CON 

0 

Temp storage | 


16. Let r be the number of terms in polynomial (M) . The subroutine requires 21pr + 
38r + 29 + 27 53 wi’ + 18 53 m" + 27 53 p' + 8 53 q' units of time, where the summations 
refer to the corresponding quantities during the r activations of Program A. The 
number of terms in polynomial(Q) goes up by p — m' each activation of Program A. If 
we make the not unreasonable assumption that m! = 0 and p' = ap where 0 < a < 1, 
we get the respective sums equal to 0, (1 - a)pr, apr , and rq' 0 + ap(r(r - l)/2), where 
q'o is the value of q in the first iteration. The grand total is 4apr 2 + 40pr + 4 apr + 
8<?o r + 38r + 29. This analysis indicates that the multiplier ought to have fewer terms 
than the multiplicand, since we have to skip over unmatching terms in polynomial (Q) 
more often. (See exercise 5.2.3-29 for a faster algorithm.) 

17. There actually is very little advantage; addition and multiplication routines with 
either type of list would be virtually the same. The efficiency of the ERASE subroutine 
(see exercise 13) is apparently the only important difference. 

18. Let the link field of node x, contain LOCCxi+i) © L0C(a;i_i), where “©” denotes 
"exclusive or.” Other invertible operations, such as addition or subtraction modulo the 
pointer field size, could also be used. It is convenient to include two adjacent list heads 
in the circular list, to help get things started properly. (The origin of this ingenious 
technique is unknown.) 

SECTION 2.2.5 

1. Insert Y at the left: P 4= AVAIL; INFO(P) 4- Y; LLINK(P) 4 - A; RLINK(P) 4 - LEFT; 
if LEFT ^ A then LLINK(LEFT) 4— P else RIGHT 4— P; LEFT 4— P. Set Y to leftmost and 
delete: if LEFT = A then UNDERFLOW; P 4- LEFT; LEFT 4- RLINK(P) ; if LEFT = A then 
RIGHT 4- A, else LLINK(LEFT) 4- A; Y 4- INFO(P); AVAIL <= P. 
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2. Consider the case of several deletions (at the same end) in succession. After each 
deletion we must know what to delete next, so the links in the list must point away 
from that end of the list. Deletion at both ends therefore implies that the links must 
go both ways. On the other hand, exercise 2.2.4-18 explains how to represent two links 
in a single link field; in that way general deque operations are possible. 

3. To show the independence of CALLUP from C ALLDOWN, notice for example that in 
Table 1 the elevator did not stop at floors 2 or 3 at time 0393-0444 although there were 
people waiting; these people had pushed CALLDOWN, but if they had pushed CALLUP the 
elevator would have stopped. 

To show the independence of CALLCAR from the others, notice that in Table 1, when 
the doors start to open at time 1378, the elevator has already decided to be GOINGUP. 
Its state would have been NEUTRAL at that point if CALLCAR [1] = CALLCAR [2] = 
CALLCAR [3] = CALLCAR [4] = 0, according to step E2, but in fact CALLCAR [2] and 
CALLCAR [3] have been set to 1 by users 7 and 9 in the elevator. (If we envision the 
same situation with all floor numbers increased by 1, the fact that STATE = NEUTRAL 
or STATE = GOINGUP when the doors open would affect whether the elevator would 
perhaps continue to go downward or would unconditionally go upward.) 

4. If a dozen or more people were getting out at the same floor, STATE might be 
NEUTRAL all during this time, and when E9 calls the DECISION subroutine this may set 
a new state before anyone has gotten in on the current floor. It happens very rarely 
indeed (and it certainly was the most puzzling phenomenon observed by the author 
during his elevator experiments). 

5. The state from the time the doors start to open at time 1063 until user 7 gets in 
at time 1183 would have been NEUTRAL, since there would have been no calls to floor 0 
and nobody on board the elevator. Then user 7 would set CALLCAR [2] <— 1 and the 
state would correspondingly change to GOINGUP. 

6. Add the condition “if OUT < IN then STATE / GOINGUP; if OUT > IN then STATE y? 
G0INGD0WN” to the condition “FLOOR = IN” in steps U2 and U4. In step E4, accept 
users from QUEUE [FLOOR] only if they are headed in the elevator’s direction, unless 
STATE = NEUTRAL (when we accept all comers). 

[Stanford’s math department has just such an elevator, but its users don’t actually 
pay much attention to the indicator lights; people tend to get on as soon as they can, 
regardless of direction. Why didn’t the elevator designers realize this, and design the 
logic accordingly by clearing both CALLUP and CALLDOWN? The whole process would be 
faster, since the elevator wouldn’t have to stop as often.] 

7. In line 227 this user is assumed to be in the WAIT list. Jumping to U4A makes sure 
that this assumption is valid. It is assumed that GIVEUPTIME is positive, and indeed 
that it is probably 100 or more. 


Comments are 

left to the reader. 

211 E8 

DEC4 

1 

218 

ENTA 

61 

219 

JMP 

H0LDC 

280 

LDA 

CALL ,4(3 : 5) 

281 

JAP 

IF 

282 

ENT1 

-2,4 

283 

J1Z 

2F 

284 

LDA 

CALL, 4(1:1) 
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285 


JAZ 

E8 



286 

2H 

LDA 

CALL- 

-1,4 


287 


ADD 

CALL- 

-2,4 


288 


ADD 

CALL- 

-3,4 


289 


ADD 

CALL- 

-4,4 


290 


JANZ 

E8 



291 

1H 

ENTA 

23 



292 


JMP 

E2A 



9. 01 

DECISION 

STJ 

9F 

Store exit location. 

02 



J5NZ 

9F 

Dl. Decision necessary? 

03 



LDX 

ELEV1+2 (NEXTINST) 


04 



DECX 

El 

D2. Should doors open? 

05 



JXNZ 

IF 

Jump if elevator not at El. 

06 



LDA 

CALL+2 


07 



ENT3 

E3 

Prepare to schedule E3, 

08 



JANZ 

8F 

if there is a call on floor 2. 

09 

1H 


ENT1 

-4 

D3. Any calls? 

10 



LDA 

CALL+4 , 1 

Search for a nonzero call variable 

11 



JANZ 

2F 


12 

1H 


INC1 

1 

ill = j — 4 

13 



J1NP 

*-3 


U 



LDA 

9F (0 : 2) 

All CALL [j] , j ^ FLOOR, are zero. 

15 



DECA 

E6B 

Is exit location = line 250? 

16 



JANZ 

9F 


17 



ENT1 

-2 

Set j 4— 2. 

18 

2H 


ENT 5 

4,1 

D4. Set STATE. 

19 



DEC5 

0,4 

STATE 4- j - FLOOR. 

20 



J5NZ 

*+2 


21 



JANZ 

IB 

j = FLOOR not allowed in general. 

22 



JXNZ 

9F 

D5. Elevator dormant? 

23 



J5Z 

9F 

Jump if not at El or if j = 2. 

24 



ENT3 

E6 

Otherwise schedule E6. 

25 

8H 


ENTA 

20 

Wait 20 units of time. 

26 



ST6 

8F(0 : 2) 

Save rI6. 

27 



ENT6 

ELEV1 


28 



ST3 

2, 6 (NEXTINST) 

Set NEXTINST to E3 or E6. 

29 



JMP 

HOLD 

Schedule the activity. 

30 

8H 


ENT6 

* 

Restore rI6. 

31 

9H 


JMP 

* 

Exit from subroutine. | 


11. Initially let LINK [A] = 0, 1 < A < n, and HEAD = — 1. During a simulation step that 
changes V[A], give an error indication if LINK [A] / 0; otherwise set LINK [A:] HEAD, 
HEAD -f- A and set NEWV[A] to the new value of V[A] . After each simulation step, set 

k 4— HEAD. HEAD < 1, and do the following operation repeatedly zero or more times 

until k < 0: set V[A] J- NEWV[A], t <- LINK [A], LINK [A] •<— 0, A 4 — t. 

Clearly this method is readily adapted to the case of scattered variables, if we 
include a NEWV and LINK field in each node associated with a variable field V. 

12. The WAIT list has deletions from the left to the right, but insertions are sorted 
in from the right to the left (since the search is likely to be shorter from that side). 
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Also we delete nodes from all three lists in several places when we do not know the 
predecessor or successor of the node being deleted. Only the ELEVATOR list could be 
converted to a one-way list, without much loss of efficiency. 

Note: It may be preferable to use a nonlinear list as the WAIT list in a discrete 
simulator, to reduce the time for sorting in. Section 5.2.3 discusses the general problem 
of maintaining priority queues, or “smallest in, first out” lists, such as this. Several 
ways are known in which only O(logn) operations are needed to insert or delete when 
there are n elements in the list, although there is of course no need for such a fancy 
method when n is known to be small. 

SECTION 2.2.6 

1. (Here the indices run from 1 to n, not from 0 to n as in Eq. (6).) L0C(A[J,K]) = 
L0C(A[0,0]) +2nJ + 2K, where A [0,0] is an assumed node that is actually nonexistent. 
If we set J = K = 1, we get L0C(A[1,1]) = L0C(A[0,0]) + 2 n + 2, so the answer can 
be expressed in several ways. The fact that L0C(A[0,0] ) might be negative has led to 
many bugs in compilers and loading routines. 

2. L0C(A [I i , . . . , I*,] ) = L0C(A[0,. ...0]) + Ei< r <*«rli- = L0C(A[Zi ,...,/*,]) + 

Ei< r < fc arlr - Ei<r<fc«r^r, where a r = cn r < s <jfc(w s -ls + 1). 

Note: For a generalization to the structures occurring in programming languages 
such as C, and a simple algorithm to compute the relevant constants, see P. Deuel, 
CACM 9 (1966), 344-347. 

3 - 1 < k < j < n if and only if 0 < k - 1 < j — 1 < n - 1; so replace fc, j, n 
respectively by k — 1, j -=* 1, n — 1 in all formulas derived for lower bound zero. 

4. L0C(A[J,K] ) = L0C(A[0,0] ) + nJ — J(J — l)/2 + K. 

5. Let AO = L0C(A[0,0] ) . There are at least two solutions, assuming that J is in 
rll and K is in rI2. (i) “LDA TA2,1:7”, where location TA2+) is “NOP j+1* j/2+A0 , 2” ; 
(ii) “LDA Cl, 7: 2”, where location Cl contains “NOP TA,1:7” and location TA +j says 
“NOP j+l*j/2+A0”. The latter takes one more cycle but doesn’t tie the table down to 
index register 2. 

6. (a) L0C(A [I , J ,K] ) = L0C(A[0,0,0] ) + + 2 ) + + X ) + 

(b) L0C(B [I , J ,K] ) = L0C(B[0,0,0] ) 

,( n + 3\ /n + 3-I\ / n + 2 — I \ /n + 2-J\ 

+ l 3 )-( 3 j + ( 2 )“( 2 ) +K ~ J ' 

hence the stated form is possible in this case also. 

7. L0C(A[I 1 ,...,I /fc ])=L0C(A[0,...,0])+y f^ + ^M.See exercise 1.2.6-56. 

l<r<fc ’ 1+k ~ V ’ 

8. (Solution by P. Nash.) Let X[I,J,K] be defined for 0 < I < n, 0 < J < n + 1, 

0 < K < n + 2. We can let A[I,J,K] = X[I,J,K]; B[I,J,K] = X[J,I + 1,K]: 
C [I , J,K] = X[I,K,J + 1]; D[I,J,K] = X [J ,K , I + 2] ; E [I , J ,K] = X [K, I + 1 , J + 1] ; 
F [I , J ,K] = X [K , J + 1,1 + 2]. This scheme is the best possible, since it packs the 
(n + l)(n + 2)(n + 3) elements of the six tetrahedral arrays into consecutive locations 
with no overlap. Proof: A and B exhaust all cells X[i,j,fc] with k — min (i,j,k)\ 
C and D exhaust all cells with j = min(i,j, k) ^ k\ E and F exhaust all cells with 

1 = min (i,j,k) / j,k. 

(The construction generalizes to m dimensions, if anybody ever wants to pack the 
elements of m! generalized tetrahedral arrays into (n + l)(n + 2) . . . ( n + m ) consecutive 
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locations. Associate a permutation 0102 . . . a m with each array, and store its elements 
in X[I ai + B\ ,1 a2 + B 2 , . . . ,Ia m + B m 2 , where B 1 B 2 . . ■ B m is an inversion table for 
ciia .2 . . . a m as defined in exercise 5. 1.1-7.) 

9. Gl. Set pointer variables PI, P2, P3, P4, P5, P6 to the first locations of the lists 
FEMALE. A21, A22, A23, BLOND, BLUE, respectively. Assume in what follows 
that the end of each list is given by link A, and A is smaller than any other 
link. If P6 = A, stop (the list, unfortunately, is empty). 

G2. (Many possible orderings of the following actions could be done; we have 
chosen to examine EYES first, then HAIR, then AGE, then SEX.) Set P5 <— 
HAIR(P5) zero or more times until P5 < P6. If now P5 < P6, go to step G5. 

G3. Set P4 AGE(P4) repeatedly if necessary until P4 < P6. Similarly do the 
same to P3 and P2 until P3 < P6 and P2 < P6. If now P4, P3, P2 are all 
smaller than P6, go to G5. 

G4. Set PI <— SEX(Pl) until PI < P6. If PI = P6, we have found one of the young 
ladies desired, so output her address, P6. (Her age can be determined from 
the settings of P2, P3, and P4.) 

G5. Set P6 <— EYES(P6) . Now stop if P6 = A; otherwise return to G2. | 

This algorithm is interesting but not the best way to organize a list for such a search. 

10. See Section 6.5. 

11. At most 200 + 200 + 3 • 4 • 200 = 2800 words. 

12. VAL(QO) = c. VAL(PO) = b/a , VAL(Pl) = d. 

13. It is convenient to have at the end of each list a sentinel that “compares low” in 
some field on which the list is ordered. A straight one-way list could have been used, for 
example by retaining just the LEFT links in BASER0W[«] and the UP links in BASEC0L [ )] , 
by modifying Algorithm S thus: In S2, test if P0 = A before setting J f- C0L(P0); if 
so, set P0 <— L0C (BASER0W [10] ) and go to S3. In S3, test if Q0 = A; if so, terminate. 
Step S4 should change by analogy with step S2. In S5, test if PI = A; if so, act as if 
C0L(P1) < 0. In S6, test if UP(PTR[J]) = A; if so, act as if its ROW field were negative. 

These modifications make the algorithm more complicated and save no storage 
space except a ROW or COL field in the list heads (which in the case of MIX is no saving 
at all). 

14. One could first link together those columns that have a nonzero element in the 
pivot row, so that all other columns could be skipped as we pivot on each row. Rows 
in which the pivot column is zero are skipped over immediately. 

15. Let rll = PIVOT. J; rI2 = P0; rI3 = Q0; rI4 = P; rI5 = PI, X; L0C(BASER0W[i] ) EE 
BROW + i; L0C(BASEC0L[j] ) = BC0L + j\ PTR[j] EE BC0L + j(l : 3). 


01 

ROW 

EQU 

0:3 


02 

UP 

EQU 

4:5 


03 

COL 

EQU 

0:3 


04 

LEFT 

EQU 

4:5 


05 

PTR 

EQU 

1:3 


06 

PIV0TSTEP 

STJ 

9F 

Subroutine entrance, rll = PIVOT 

07 

SI 

LD2 

0,1 (ROW) 

SI. Initialize. 

08 


ST2 

10 

10 ROW(PIVOT) . 

09 


LD3 

1 , 1 (COL) 
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10 


ST3 

JO 

11 


LDA 

=1.0= 

12 


FDIV 

2,1 

13 


STA 

ALPHA 

14 


LDA 

= 1.0= 

15 


STA 

2,1 

16 


ENT2 

BROW, 2 

17 


ENT3 

BC0L,3 

18 


JMP 

S2 

19 

2H 

ENTA 

BCOL, 1 

20 


STA 

BCOL , 1 (PTR) 

21 


LDA 

2,2 

22 


FMUL 

ALPHA 

23 


STA 

2,2 

24 

S2 

LD2 

1,2 (LEFT) 

25 


LD1 

1 ,2(C0L) 

26 


JINN 

2B 

27 

S3 

LD3 

0,3 (UP) 

28 


LD4 

0,3 (ROW) 

29 

9H 

J4N 

* 

30 


CMP4 

10 

31 


JE 

S3 

32 


ST4 

I (ROW) 

33 


ENT4 

BROW, 4 

34 

S4A 

LD5 

1,4 (LEFT) 

35 

S4 

LD2 

1,2 (LEFT) 

36 


LD1 

1 ,2 (COL) 

37 


CMP1 

JO 

38 


JE 

S4 

39 


ENTA 

0,1 

40 


SLA 

2 

41 


JINN 

S5 

42 


LDAN 

2,3 

43 


FMUL 

ALPHA 

44 


STA 

2,3 

45 


JMP 

S3 

46 

1H 

ENT4 

0,5 

45 


LD5 

1,4 (LEFT) 

48 

S5 

CMPA 

1 , 5(C0L) 

49 


JL 

IB 

50 


JE 

S7 

51 

S6 

LD5 

BCOL, 1 (PTR) 

52 


LDA 

I 

53 

2H 

ENT6 

0,5 

54 


LD5 

0,6 (UP) 

55 


CMPA 

0,5 (ROW) 

56 


JL 

2B 

57 


LD5 

AVAIL 

58 


J5Z 

OVERFLOW 


JO 4- COL (PIVOT). 
Floating point constant 1 

ALPHA 4- 1/VAL(PIV0T) . 

VAL (PIVOT) 4- 1. 

PO 4- LOC (BASEHOW [10] ) . 
QO 4- LOC (BASECOL [ JO] ) . 


PTR[J] 4- LOC (BASECOL [J] ) . 


VAL(PO) -f- ALPHA x VAL(PO). 

52. Process pivot row. PO 4— LEFT(PO). 
J 4- COL(PO) . 

If J > 0, process J. 

53. Find new row. QO 4— UP(QO). 
rI4 4- ROW(QO). 

If rI4 < 0, exit. 

If rI4 = 10, repeat. 

I 4- rI4. 

P 4- LOC (BASEROW [I] ) . 

PI 4- LEFT (P) . 

54. Find new column. PO 4— LEFT(PO). 
J 4- COL(PO) . 

Repeat if J = JO. 

rA(0 : 3) 4- J. 

If J < 0, 

set VAL(QO) 4 ALPHA x VAL(QO). 


P 4- PI. 

PI 4- LEFT (P) . 

55. Find I, J element. 

Loop until COL (PI) < J. 

If =, go right to S7. 

56. Insert I, J element. rI5 4— PTR[J], 
rA(0 : 3) 4- I. 

rI6 4— rI5. 
rI5 4- UP(rI6). 

Jump if R0W(rI5) > I. 

X AVAIL. 
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59 

LDA 

0,5 (UP) 


60 

STA 

AVAIL 


61 

LDA 

0,6 (UP) 


62 

STA 

0,5 (UP) 

UP(X) UP (PTR [J] ) . 

63 

LDA 

1,4 (LEFT) 


64 

STA 

1,5 (LEFT) 

LEFT(X) 4- LEFT(P) . 

65 

ST1 

1 , 5 (COL) 

COL(X) 4- J. 

66 

LDA 

I (ROW) 


67 

STA 

0,5 (ROW) 

ROW (X) 4- I. 

68 

STZ 

2,5 

VAL(X) 4- 0. 

69 

ST5 

1,4 (LEFT) 

LEFT(P) -6- X. 

70 

ST5 

0,6 (UP) 

UP (PTR [J]) 4- X. 

71 S7 

LDAN 

2,3 

S7. Pivot. -VAL(QO) 

72 

FMUL 

2,2 

X VAL(PO) 

73 

FADD 

2,5 

+ VAL(Pl). 

74 

JAZ 

S8 

If significance lost, to S8. 

75 

STA 

2,5 

Otherwise store in VAL(Pl) . 

76 

ST5 

BC0L,1(PTR) 

PTR[J] 4- PI. 

77 

ENT4 

0,5 

P 4- PI. 

78 

JMP 

S4A 

PI 4- LEFT(P) , to S4. 

00 

m 

o> 

LD6 

BCOL , 1 (PTR) 

S8. Delete I. J element. rI6 4— PTR[J] . 

80 

JMP 

*+2 


81 

LD6 

0,6 (UP) 

rI6 4— UP(rI6) . 

82 

LDA 

0,6 (UP) 


83 

DECA 

0,5 

Is UP(rI6) = PI? 

84 

JANZ 

*-3 

Loop until equal. 

85 

LDA 

0,5 (UP) 


86 

STA 

0,6 (UP) 

UP(rI6) 4- UP(P1). 

87 

LDA 

1,5 (LEFT) 


88 

STA 

1,4 (LEFT) 

LEFT(P) 4— LEFT(Pl) . 

89 

LDA 

AVAIL 

AVAIL -<= PI. 

90 

STA 

0,5 (UP) 


91 

ST5 

AVAIL 


92 

JMP 

S4A 

PI 4- LEFT (P), to S4. | 


Note: Using the conventions of Chapter 4, lines 71-74 would actually be coded 


LDA 2,3; FMUL 2,2; FCMP 2,5; JE S8; STA TEMP; LDA 2,5; FSUB TEMP; 
with a suitable parameter EPSILON in location zero. 

17. For each row i and each element A[i,fc] ^ 0, add A[i,A-] times row k of B to row i 
of C. Maintain only the COL links of C while doing this; the ROW links are easily filled in 
afterwards. [A. Schoor, Inf. Pioc. Letters 15 (1982), 87-89.] 

18. The three pivot steps, in respective columns 3, 1, 2, yield respectively 


/ 

1 

3 

2 

3 

3 \ 

( ^ 

1 

2 


t 

0 

1 

°\ 

- 

2 

3 

1 

3 

"I > 

"I 

1 

2 

1 • 


-2 

1 

1 

V- 

1 

3 

2 

3 


u 

1 

2 

0/ 

\ 

1 

-2 

0/ 
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after the final permutations, we have the answer 


1 -2 1 
0 1 -2 
0 0 1 

20. ao = L0C(A [1 , 1] ) — 3, o-i = 1 or 2, 02 = 3 — a\ . 

21. For example, M max(I, J), L0C(A[I,J]) = L0C(A[1,1]) + M(M - 1) + I - J. 
(Such formulas have been proposed independently by many people. A. L. Rosen- 
berg and H. R. Strong have suggested the following fc-dimensional generalization: 
L0C(A[Ii , . . . ,Ifc] ) as!*, where Li = L0C(A[1 , ... , 1] ) + Ii — 1, L r = L r _j +(M r - l) r + 
(M r — Ir) (Mr 1 ~ (Mr — l) r x ), and M,. = max(Ii, . . . , I r ) [IBM Technical Disclosure 
Bulletin 14 (1972), 3026-3028]. See Current Trends in Programming Methodology 4 
(Prentice-Hall, 1978), 263-311, for further results of this kind.) 

22. According to the combinatorial number system (exercise 1.2.6-56), we can let 



« **>-(', ‘M i,+ ? +1 )+-+( 


ii + *2 + ■ 


■ ■ + ik + k — 1 
k 


[Det, Kongelige Norske Videnskabers Selskabs Forhandlinger 34 (1961), 8-9.] 

23. Let c[J] = LOC (A [0 , J] ) = L0C(A [0,0]) +raJ, if there were m rows when the matrix 
grew from J to J + 1 columns; similarly, let r[l] = LOC (A [1,0]) = LOC (A [0,0]) +nl, if 
there were n columns when we created row I. Then we can use the allocation function 


LOC (A [I, J]) 


I + c[j], if c[J] > r[l]; 

J + r[l], otherwise. 


It is not hard to prove that c[J] > r[l] implies c[J] > r[l] + J, and c[J] < r[l] implies 
c[J] + I < r[l]; therefore the relation 


L0C(A [I , J] ) = max(l + LOC (A [0, J] ) , J + LOC (A [1,0])) 

also holds. We need not restrict allocation to mn consecutive locations; the only 
constraint is that, when the matrix grows, we allocate m or n consecutive new cells in 
locations greater than those previously used. This construction is due to E. J. Otoo 
and T. H. Merrett [Computing 31 (1983), 1-9], who also generalized it to k dimensions. 

24. [Aho, Hopcroft, and Ullman, The Design and Analysis of Computer Algorithms 
(Addison- Wesley, 1974), exercise 2.12.] Besides the array A, maintain also a verification 
array V of the same size, and a list L of the locations used. Let n be the number of 
items in L; initially n = 0 and the contents of L. A, and V are arbitrary. Whenever you 
want to access A [fc] for a value of k that you might not have used before, first check 
whether 0 < V[fc] < n and L[V[fc]] = k. If not, set V[A0 <- n , L[n] «- k, A [A:] «- 0. 
and n <- n+ 1. Otherwise you can be sure that A[fc] already contains legitimate data. 
(By a slight extension of this method, it is possible to save and eventually restore the 
contents of all entries of A and V that change during the computation.) 


SECTION 2.3 

1. There are three ways to choose the root. Once the root has been chosen, say .4, 
there are three ways to partition the other nodes into subtrees: {B}, {C}; {C}, { S}: 
{B,C}. In the latter case there are two ways to make {B,C} into a tree, depending 
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on which is the root. Hence we get 
the four trees shown when A is the 
root, and 12 in all. This problem is 
solved for any number n of nodes 
in exercise 2.3.4.4-23. 

2 . The first two trees in the answer to exercise 1 are the same, as oriented trees, so 
we get only 9 different possibilities in this case. For the general solution, see Section 
2. 3. 4. 4, where the formula n n ~ 1 is proved. 

3. Part 1: To show there is at least one such sequence. Let the tree have n nodes. The 
result is clear when n = 1, since A must be the root. If n > 1, the definition implies 
there is a root X\ and subtrees Ti,T 2 ,. . . , T m ; either X = X\ or X is a member of a 
unique Tj. In the latter case, there is by induction a path X 2 , ■ ■ ■ , X where X 2 is the 
root of Tj, and since Xi is the parent of X 2 we have a path Xi, X 2 , . . . , X. 

Part 2: To show there is at most one such sequence. We will prove by induction 
that if X is not the root of the tree, A' has a unique parent (so that Xk determines Xk-i 
determines AA- 2 , etc.) If the tree has one node, there is nothing to prove; otherwise 
X is in a unique Tj. Either X is the root of Tj, in which case X has a unique parent 
by definition; or X is not the root of Tj, in which case X has a unique parent in Tj by 
induction, and no node outside of Tj can be A’s parent. 

4. True (unfortunately). 

5. 4. 

6. Let parent^ (A) denote X, and let parentf fc+1 ^ (A) = parent (parent^ (A)), so 
that parent^ (A') is A’s parent, and parent^ (A) is A’s grandparent; when k > 2, 
parent^ (A) is A’s “(great-) fc_2 grandparent.” The requested cousinship condition is 
that parent^ m+1 *(A) = parent^ m+n+1 l (Y) but parent^ (A) ^ parent^ m+n ' (Y ). When 
n > 0, this relation is not symmetrical with respect to A and Y , although people 
usually treat it as symmetrical in everyday conversation. 

7. Use the (unsymmetric) condition defined in exercise 6, with the convention that 
parent^' (A) ^ parent^ (F) if either j or k (or both) is —1. To show that this relation 
is always valid for some unique m and n, consider the Dewey decimal notation for A 
and Y . namely l.ai. • • • ,a p .b\. ■ ■ ■ ,b q and l.ai. • • • .a p .Ci. • • • ,c r , where p > 0, q > 0, 
r > 0 and (if qr / 0) b\ ^ ci. The Dewey numbers of any pair of nodes can be written 
in this form, and clearly we must take m = q — 1 and to + n = r — 1. 

8. No binary tree is really a tree; the concepts are quite separate, even though the 
diagram of a nonempty binary tree may look treelike. 

9 . A is the root, since we conventionally put the root at the top. 

10. Any finite collection of nested sets corresponds to a forest as defined in the text, as 
follows: Let Ai, . . . , A„ be the sets of the collection that are contained in no other. For 
fixed j, the sub-collection of all sets contained in Aj is nested; hence we may assume 
that this sub-collection corresponds to a tree (unordered) with Aj as the root. 

11. In a nested collection C let A = Y if there is some Z G C such that A U Y C Z. 
This relation is obviously reflexive and symmetric, and it is in fact an equivalence 
relation since W = X and A = Y implies that there are Z\ and Z 2 in C with W C Z \ , 
A' C Zi n Zo, and F C Z 2 . Since Zi n Z 2 / 0, either Z\ C Z 2 or Z 2 U Z\\ hence 
W U Y C Zi U Z 2 € C. Now if C is a nested collection, define an oriented forest 
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corresponding to C by the rule “ X is an ancestor of Y, and Y is a descendant of A", if 
and only if X D Y” Each equivalence class of C corresponds to an oriented tree, which is 
an oriented forest with X = Y for all X, Y. (We thereby have generalized the definitions 
of forest and tree that were given for finite collections.) In these terms, we may define 
the level of X as the cardinal number of ancestors(A). Similarly, the degree of X is 
the cardinal number of equivalence classes in the nested collection descendants) A). We 
say X is the parent of Y, and Y is a child of X, if X is an ancestor of Y but there is 
no Z such that X D Z D Y. (It is possible for X to have descendants but no children, 
ancestors but no parent.) To get ordered trees and forests, order the equivalence classes 
mentioned above in some ad hoc manner, for example by embedding the relation C 
into linear order as in exercise 2.2.3-14. 

Example (a): Let S ak = { x \ x = .d\d 2 d 3 . . . in decimal notation, where a = 
.e 1 e 2 e 3 ... in decimal notation, and dj = e 3 if j mod 2 k / 0}. The collection C = 
{S a k [ fc > 0, 0<a<l}is nested, and gives a tree with infinitely many levels and 
uncountable degree for each node. 

Example (b), (c): It is convenient to define this set in the plane, instead of in terms 
of real numbers, and this is sufficient since there is a one-to-one correspondence between 
the plane and the real numbers. Let S amn = {(a,y) | m/2 n < y < (m + l)/2"}, 
and let T a = {( x,y ) i < a}. The collection C = {S amn | 0 < a < 1, n > 0, 
0 < m < 2 n } U {T a | 0 < a < 1} is easily seen to be nested. The children of 
Samn are S a ( 2 m )(n+i) and S a ( 2 m +i)( n +i), and T a has the child S a oo plus the subtree 
{5,3 mn I P < «} u {Tp | P < a}. So each node has degree 2, and each node has 
uncountably many ancestors of the form T a . This construction is due to R. Bigelow. 

Note: If we take a suitable well-ordering of the real numbers, and if we define 
T a = {(x,y) | x y a}, we can improve this construction slightly, obtaining a nested 
collection where each node has uncountable level, degree 2, and two children. 

12. We impose an additional condition on the partial ordering (analogous to that of 
“nested sets”) to ensure that it corresponds to a forest: If x ■< y and x <z then either 
y X z or z A y. In other words, the elements larger than any given element are linearly 
ordered. To make a tree, also assert the existence of a largest element r such that x < r 
for all x. A proof that this gives an unordered tree as defined in the text, when the 
number of nodes is finite, runs like the proof for nested sets in exercise 10. 

13. a\, ai.d 2 , ..., ai. 02 .---.afc. 

14. Since S is nonempty, it contains an element I . n, . ■ ■ ■ . a k where k is as small as 
possible; if k > 0 we also take a k as small as possible in S, and we immediately see that 
k must be 0. In other words, 5 must contain the element 1. Let 1 be the root. All other 
elements have k > 0, and so the remaining elements of 5 can be partitioned into sets 
Sj = {l.j.a. 2 - ■ ■ ■ -a k }, 1 < j < m, for some m > 0. If m / 0 and S m is nonempty, we 
deduce by reasoning as above that 1 .j is in Sj for each Sj; hence each Sj is nonempty. 
Then it is easy to see that the sets Sj = {l.a 2 . • • • .a. k \ \.j.a 2 - ■ ■ ■ .a k is in Sj } satisfy 
the same condition as S did. By induction, each of the Sj forms a tree. 

15. Let the root be 1, and let the roots of the left and right subtrees of a be o.O 
and a. 1, respectively, when such roots exist. For example, King Christian IX appears 
in two positions of Fig. 18(a), namely 1.0. 0.0.0 and 1.1. 0.0. 1.0. For brevity we may 
drop the decimal points and write merely 10000 and 110010. Note: This notation is 
due to Francis Galton; see Natural Inheritance (Macmillan, 1889), 249. For pedigrees, 
it is more mnemonic to use F and M in place of 0 and f and to drop the initial 1; 
thus Christian IX is Charles’s MFFMF, his mother’s father’s father’s mother’s father. 
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The 0 and 1 convention is interesting for another reason: It gives us an important 
correspondence between nodes in a binary tree and positive integers expressed in the 
binary system (namely, memory addresses in a computer). 



17 . parent(Z[l]) = A; parent(Z[l, 2]) = C; parent(Z[l, 2, 2]) = E. 


18 . L[ 5, 1, 1] = (2). L[3, 1 ] is nonsense, since L[3] is an empty List. 


19 . *[L] L[2] = (L); L[2, 1 , 1 ] = a. 

/ \ 


a 


* 


[L] 

20 . (Intuitively, the correspondence between 0-2-trees and binary trees is obtained by 
removing all terminal nodes of the 0 -2-tree; see the important construction in Section 
2. 3. 4. 5.) Let a 0-2-tree with one node correspond to the empty binary tree; and let 
a 0 -2-tree with more than one node, consisting therefore of a root r and 0 -2-trees Ti 
and T 2 , correspond to the binary tree with root r, left subtree T[, and right subtree T 2 , 
where Ti and T 2 correspond respectively to T[ and T 2 - 

21. 1+0 • ni + 1 • ri 2 + • • • + (m — 1) • n m . Proof: The number of nodes in the tree 
is no + m + n 2 + • • • + n m , and this also equals 1 + (number of children in the tree) 
= 1 + 0 • n 0 + 1 • ni + 2 • n 2 + • • • + m ■ n m - 

22 . The basic idea is to proceed recursively, with the representation of a nonempty 
binary tree defined to be the representation of its root plus half-size-and-rotated rep- 
resentations of its left and right subtrees. Thus an arbitrarily large binary tree can be 
represented on a single sheet of paper, if one has a sufficiently powerful magnifying glass. 

Many variations on this theme are possible. For example, one idea is to represent 
the root by a line from the center of a given landscape-oriented page to the top edge, 
and to rotate the left-subtree representation by 90° clockwise in the left halfpage, the 
right-subtree representation by 90° counterclockwise in the right halfpage. Each node 
is then represented by a line. (When this method is applied to a complete binary tree 
having 2 k — 1 nodes on k levels, it yields so-called “H-trees,” which are the most efficient 
layouts of such binary trees on a VLSI chip; see R. P. Brent and H. T. Kung, Inf. Proc. 
Letters 11 (1980), 46-48.) 
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Another idea is to represent an empty binary tree by some sort of box, and to rotate the 
subtree representations of nonempty binary trees so that left subsubtrees are alternately 
to the left of or below the corresponding right subsubtrees, depending on whether the 
depth of recursion is even or odd. Then the boxes correspond to external nodes in 
an extended binary tree (see Section 2.3.4.5). This representation, which is strongly 
related to the 2-d trees and quadtrees discussed in Section 6.5, is especially appropriate 
when the external nodes carry information but the internal nodes do not. 

SECTION 2.3.1 

1. INFO(T) = A, INFO (RLINK (T) ) = C, etc.; the answer is H. 

2. Preorder: 1245367; symmetric order: 4251637; postorder: 4526731. 

3. The statement is true; notice, for example, that nodes 4, 5, 6, 7 always appear in 
this order in exercise 2. The result is immediately proved by induction on the size of 
the binary tree. 

4. It is the reverse of postorder. (This is easily proved by induction.) 

5. In the tree of exercise 2, for example, preorder is 1, 10, 100, 101, 11, 110, 111, using 
binary notation (which is in this case equivalent to the Dewey system). The strings of 
digits have been sorted, like words in a dictionary. 

In general, the nodes will be listed in preorder if they are sorted lexicographically 
from left to right, with “blank” < 0 < 1. The nodes will be listed in postorder if they 
are sorted lexicographically with 0 < 1 < “blank”. For inorder, use 0 < “blank” < 1.' 

(Moreover, if we imagine the blanks at the left and treat the Dewey labels as 
ordinary binary numbers, we get level order-, see 2.3.3-(8).) 

6 . The fact that P1P2 ■ ■ ■ p n is obtainable with a stack is readily proved by induction 
on n, or in fact we may observe that Algorithm T does precisely what is required in 
its stack actions. (The corresponding sequence of S’s and X’s, as in exercise 2.2. 1-3, is 
the same as the sequence of Is and 2s as subscripts in double order; see exercise 180 

Conversely, if pi p 2 . . . p n is obtainable with a stack and if p k = 1 , then pi . . . p k _ 1 is 
a permutation of {2, ... , k} and p k+1 ... p n is a permutation of {fc + 1, . . . , n}; these are 
the permutations corresponding to the left and right subtrees, and both are obtainable 
with a stack. The proof now proceeds by induction. 

7. From the preorder, the root is known; then from the inorder, we know the left 
subtree and the right subtree; and in fact we know the preorder and inorder of the 
nodes in the latter subtrees. Hence the tree is readily constructed (and indeed it 
is quite amusing to construct a simple algorithm that links the tree together in the 
normal fashion, starting with the nodes linked together in preorder in LLINK and in 
inorder in RLINK). Similarly, postorder and inorder together characterize the structure. 
But preorder and postorder do not; there are two binary trees having AB as preorder 
and BA as postorder. If all nonterminal nodes of a binary tree have both branches 
nonempty, its structure is characterized by preorder and postorder. 

8. (a) Binary trees with all LLINKs null, (b) Binary trees with zero or one nodes, 
(c) Binary trees with all RLINKs null. 

9. T1 once, T2 2n+l times, T3 n times, T4 n+ 1 times, T5 n times. These counts 
can be derived by induction or by Kirchhoff’s law, or by examining Program T. 

10. A binary tree with all RLINKs null will cause all n node addresses to be put in the 
stack before any are removed. 
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11. Let a„k be the number of binary trees with n nodes for which the stack in 
Algorithm T never contains more than k items. If gk{z) = a n k z n , we find gi(z) = 
1/(1 -«), 92(2) = 1/(1- 2/(1-*)) = (1 — *)/(! — 2z), ...,g k (z) = l/(l - zg k -i{zj) = 
Qk-i{z)/qk{z) where <7-1(2) = q 0 (z) = 1, ^+1(2) = q k {z) - zq k -i(z)-, hence g k (z) = 
(fi(z) k+1 - f 2 (z) k+1 )/(h(z) k + 2 - f 2 (z) k+2 ) where /,(z) = |(l ± VT ~z). It can 
now be shown that a nk = [u n ] (1 - u)(l + u) 2n ( 1 - u k+1 )/( 1 - u k+2 ); hence s n = 
Efe>i k(a,nk ~ On(fc-i)) is K +1 ] (1 - u) 2 (l + u) 2n X)j>i W J /(1 - w J ), minus a nn . The 
technique of exercise 5.2.2-52 now yields the asymptotic series 


Sn/a-nn = \fnn - 


13 

24 


l + ' + 0 ( n- 3 ' 2 ). 

n 2 n 


[N. G. de Bruijn, D. E. Knuth, and S. O. Rice, in Graph Theory and Computing , ed. 
by R. C. Read (New York: Academic Press, 1972), 15-22.] 

When the binary tree represents a forest as described in Section 2.3.2, the quantity 
analyzed here is the height of that forest (the furthest distance between a node and a 
root, plus one). Generalizations to many other varieties of trees have been obtained by 
Flajolet and Odlyzko [J. Computer and System Sci. 25 (1982), 171-213]; the asymptotic 
distribution of heights, both near the mean and far away, was subsequently analyzed by 
Flajolet, Gao, Odlyzko, and Richmond [ Combinatorics , Probability, and Computing 2 
(1993), 145-156], 

12. Visit NODE(P) between steps T2 and T3, instead of in step T5. For the proof, 
demonstrate the validity of the statement il Starting at step T2 with . . . original value 
A [1] ... A[m] ,” essentially as in the text. 

13. (Solution by S. Araujo, 1976.) Let steps T1 through T4 be unchanged, except 
that a new variable Q is initialized to A in step Tl; Q will point to the last node visited, 
if any. Step T5 becomes two steps: 

T5. [Right branch done?] If RLINK(P) = A or RLINK(P) = Q, go on to T6; 

otherwise set A <= P, P <— RLINK(P) and return to T2. 

T6. [Visit P.] Visit NODE(P), set Q t— P, and return to T4. 

A similar proof applies. (Steps T4 and T5 can be streamlined so that nodes are not 
taken off the stack and immediately reinserted.) 

14. By induction, there are always exactly n + 1 A links (counting T when it is null). 
There are n nonnull links, counting T, so the remark in the text about the majority of 
null links is justified. 

15. There is a thread LLINK or RLINK pointing to a node if and only if it has a nonempty 
right or left subtree, respectively. (See Fig. 24.) 

16. If LTAG (Q) = 0, Q* is LLINK (Q); thus Q* is one step down and to the left. Otherwise 
Q* is obtained by going upwards in the tree (if necessary) repeatedly until the first time 
it is possible to go down to the right without retracing steps; typical examples are the 
trips from P to P* and from Q to Q* in the following tree: 
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17. If LTAG(P) — 0, set Q 4— LLINK(P) and terminate. Otherwise set Q 4 — P, then set 
Q 4- RLINK(Q) zero or more times until Bnding RTAG(Q) = 0; finally set Q 4 - RLINK(Q) 
once more. 

18. Modify Algorithm T by inserting a step T2.5, “Visit NODE(P) the first time”; in 
step T5, we are visiting NODE(P) the second time. 

Given a threaded tree the traversal is extremely simple: 

(P, 1) = (LLINK(P) , 1) if LTAG(P) = 0, otherwise (P, 2); 

(P,2) A = (RLINK (P) , 1) if RTAG(P) =0, otherwise (RLINK(P) . 2). 

In each case, we move at most one step in the tree; in practice, therefore, double order 
and the values of cl and e are embedded in a program and not explicitly mentioned. 

Suppressing all the first visits gives us precisely Algorithms T and S; suppressing 
all the second visits gives us the solutions to exercises 12 and 17. 

19. The basic idea is to start by finding the parent Q of P. Then if P / LLINK(Q) we 
have Pc = Q; otherwise we can find PC by repeatedly setting Q 4- Q$ zero or more times 
until RTAG(Q) = 1. (See, for example, P and Pc in the tree shown.) 



There is no efficient algorithm to find the parent of P in a general right-threaded 
tree, since a degenerate right-threaded tree in which all left links are null is essentially a 
circular list in which the links go the wrong way. Therefore we cannot traverse a right- 
threaded tree in postorder with the same efficiency as the stack method of exercise 13, 
if we keep no history of how we have reached the current node P. 

But if the tree is threaded in both directions, we can find P’s parent efficiently: 

FI. Set Q 4— P and R 4— P. 

F2. If LTAG(Q) = RTAG(R) = 0, set Q 4- LLINK(Q) and R 4 - RLINK (R) and repeat 
this step. Otherwise go to F4 if RTAG(R) =1. 

F3. Set Q 4 LLINK(Q) , and terminate if P = RLINK (Q) . Otherwise set R 4 - 
RLINK(R) zero or more times until RTAG(R) = 1, then set Q 4 - RLINK (R) and 
terminate. 

F4. Set R 4- RLINK (R) , and terminate with Q 4— R if P = LLINK(R) . Otherwise set 
Q 4- LLINK(Q) zero or more times until LTAG(Q) = 1, then set Q 4- LLINK(Q) 
and terminate. | 

The average running time of Algorithm F is 0(1) when P is a random node of the tree. 
For if we count only the steps Q 4— LLINK(Q) when P is a right child, or only the steps 
R 4— RLINK (R) when P is a left child, each link is traversed for exactly one node P. 
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20. Replace lines 06-09 by: Replace lines 12-13 by: 


ENT4 

0,6 

LD4 

0,6 (LINK) 

LD6 

AVAIL 

LD5 

0,6 (INFO) 

J6Z 

OVERFLOW 

LDX 

AVAIL 

LDX 

0,6 (LINK) 

STX 

0,6 (LINK) 

STX 

AVAIL 

ST6 

AVAIL 

ST5 

0,6(INF0) 

ENT6 

0,4 

ST4 

0,6(LINK) 




If two more lines of code are added at line 06 
T3 LD3 0,5(LLINK) 

J3Z T5 To T5 if LLINK(P) = A. 

with appropriate changes in lines 10 and 11, the running time goes down from (30n + 
a + 4 )u to (27a + 6n — 22)u. (This same device would reduce the running time of 
Program T to (12a + 6n — 7)u, which is a slight improvement, if we set a = (n + l)/2.) 
21 . The following solution by Joseph M. Morris [In/. Proc. Letters 9 (1979), 197-200] 
traverses also in preorder (see exercise 18). 

Ul. [Initialize.] Set P <— T and R <— A. 

U2. [Done?] If P = A, the algorithm terminates. 

U3. [Look left.] Set Q <- LLINK(P) . If Q = A, visit N0DE(P) in preorder and go 
to U6. 

U4. [Search for thread.] Set Q <— RLINK(Q) zero or more times until either Q = R 
or RLINK (Q) = A. 

U5. [Insert or remove thread.] If Q ^ R, set RLINK(Q) P and go to U8. Otherwise 
set RLINK (Q) <— A (it had been changed temporarily to P, but we’ve now 
traversed P’s left subtree). 

U6. [Inorder visit.] Visit N0DE(P) in inorder. 

U7. [Go to right or up.] Set R<— P, Pf- RLINK(P) , and return to U2. 

U8. [Preorder visit.] Visit N0DE(P) in preorder. 

U9. [Go to left.] Set P <— LLINK(P) and return to step U3. | 

Morris also suggested a slightly more complicated way to traverse in postorder. 

A completely different solution was found by J. M. Robson [Inf. Proc. Letters 2 
(1973), 12-14]. Let’s say that a node is “full” if its LLIMK and RLINK are nonnull, 
“empty” if its LLINK and RLINK are both empty. Robson found a way to maintain a 
stack of pointers to the full nodes whose right subtrees are being visited, using the link 
fields in empty nodes! 

Yet another way to avoid an auxiliary stack was discovered independently by G. 
Lindstrom and B. Dwyer, Inf. Proc. Letters 2 (1973), 47-51, 143-145. Their algorithm 
traverses in triple order — it visits every node exactly three times, once in each of 
preorder, inorder, and postorder — but it does not know which of the three is currently 
being done. 

Wl. [Initialize.] Set P «— T and Q «— S, where S is a sentinel value — any number 
that is known to be different from any link in the tree (e.g., —1). 

W2. [Bypass null.] If P = A, set P «— Q and Q «— A. 

W3. [Done?] If P = S, terminate the algorithm. (We will have Q = T at termina- 
tion.) 
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W4. [Visit.] Visit NODE(P). 

W5. [Rotate.] Set R <- LLINK(P) , LLINK(P) <- RLINK (P) , RLINK(P) «- Q, Q p. 
P R, and return to W2. | 

Correctness follows from the fact that if we start at W2 with P pointing to the root of 
a binary tree T and Q pointing to X, where X is not a link in that tree, the algorithm 
will traverse the tree in triple order and reach step W3 with P = X and Q = T. 

If o?(T) — X1X2 . . ■ X3 n is the resulting sequence of nodes in triple order, we have 
«(T) = T a(LLINK(T)) T q(RLINK(T)) T. Therefore, as Lindstrom observed, the three 
subsequences X1X4 . . . X3 n -2, X2X5 ■ ■ ■ ®3n-i, X3Xe . . . X3 n each include every tree node 
just once. (Since Tj+i is either the parent or child of Xj, these subsequences visit the 
nodes in such a way that each is at most three links away from its predecessor. Section 
7.2. 1.6 describes a general traversal scheme called prepostorder that has this property 
not only for binary trees but for trees in general.) 

22. This program uses the conventions of Programs T and S, with Q in rI6 and/or rI4. 
The old-fashioned MIX computer is not good at comparing index registers for equality, 


so variable R is omitted and the test “Q = R” is 

changed to “RLINK (Q) =P”. 

01 

U1 

LD5 

HEAD(LLINK) 

1 

Ul. Initialize. P T. 

02 

U2A 

J5Z 

DONE 

1 

Stop if p = A. 

03 

U3 

LD6 

0,5(LLINK) 

n + a — 1 

U3. Look left. 0 LI.TNK (Pi 

04 


J6Z 

U6 

n + a — 1 

To U6 if Q = A. 

05 

U4 

CMP5 

1,6 (RLINK) 

2 n — 2b 

U4. Search for thread. 

06 


JE 

5F 

2 n — 2b 

Jump if RLINK (q) = P. 

07 


ENT4 

0,6 

2n — 2b — a + l 

rI4 <- q. 

08 


LD6 

1,6 (RLINK) 

2n — 26 — a+ 1 


09 


J6NZ 

U4 

2n — 26 — a+ 1 

To U4 with q t- RLINK (q) if it’s ^ 0. 

10 

U5 

ST5 

1,4 (RLINK) 

a — 1 

U5 a. Insert thread. RLINK (O') <— P 

11 

U9 

LD5 

0,5(LLINK) 

a — 1 

U9. Go to left. P <- LI.TNK (PI 

12 


JMP 

U3 

a — 1 

To U3. 

13 

5H 

STZ 

1,6 (RLINK) 

a — 1 

U5b. Remove thread. RLINK (O') A 

14 

U6 

JMP 

VISIT 

n 

U6. Inorder visit. 

15 

U7 

LD5 

1,5 (RLINK) 

n 

U7. Go to right or no. P <- RLINK (P) 

16 

U2 

J5NZ 

U3 

n 

U2. Done? To TJ3 if P ^ A 

17 

DONE 




1 


The total running time is 21n + 6a - 3 - 146, where n is the number of nodes, a is 
the number of null RLINKs (hence a - 1 is the number of nonnull LLINKs), and b is the 
number of nodes on the tree’s “right spine” T, RLINK(T) , RLINK (RLINK (T) ) , etc. 

23. Insertion to the right: RLINKT(Q) «- RLINKT(P) , RLINK (P) <- Q, RTAG(P) <- 0. 
LLINK(Q) «- A. Insertion to the left, assuming LLINK(P) = A: Set LLINK(P) <- Q. 
LLINK(q) <— A, RLINK(Q) <— P, RTAG(Q) <— 1. Insertion to the left, between P and 
LLINK(P) / A: Set R «- LLINK(P) , LLINK(Q) <- R, and then set R «- RLINK (R) zero or 
more times until RTAG(R) = 1; finally set RLINK (R) <- Q, LLINK(P) <- Q. RLINK(Q) <- P 
RTAG(Q) <- 1. 

(A more efficient algorithm for the last case can be used if we know a node F 
such that P = LLINK(F) or P = RLINK (F) ; assuming the latter, for example, we could 
set INFO(P) o INFO (Q) , RLINK (F) <- Q, LLINK(Q) <- P, RLINKT(Q) <- RLINKT(P) . 
RLINK (P) «- q, RTAG(P) <- 1; this takes a fixed amount of time, but it is generally not 
recommended because it switches nodes around in memory.) 
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24. No: 




25. We first prove (b), by induction on the number of nodes in T, and similarly (c). 
Now (a) breaks into several cases; write T T' if (i) holds, T -<2 T' if (ii) holds, etc. 
Then T^i T' and T' S T" implies T ^1 T" ; T T' and T' < T" implies T ^ 2 T"; 
and the remaining two cases are treated by proving (a) by induction on the number of 
nodes in T. 

26. If the double order of T is (ui,di), (u 2 , d 2 ), • • ■ , (u 2n , d 2 n) where the u’s are nodes 

and the d's are 1 or 2, form the “trace” of the tree (vi, si), (u 2 , s 2 ), • • • , (v 2 n, s 2n ), where 
Vj = info(Mj), and Sj = l(uj) or r(uj) according as dj = 1 or 2. Now T ■< T’ if and only 
if the trace of T (as defined here) lexicographically precedes or equals the trace of T'. 
Formally, this means that we have either n < n 1 and = ( u' , s) ) for 1 < j < 2 n, 

or else there is a k for which ( Vj,Sj ) = (r '• . .s' ) for 1 < j < k and either v k < v' k or 
v k = v' k and s k < s' k . 

27. Rl. [Initialize.] Set P HEAD, P' <— HEAD'; these are the respective list heads of 

the given right-threaded binary trees. Go to R3. 

R2. [Check INFO.] If INFO(P) -<I INFO(P'), terminate (T -< T'); if INFO(P) ^ 
INFO(P'), terminate (T y T'). 

R3. [Go to left,.] If LLINK(P) = A = LLINK(P'), go to R4; if LLINK(P) = A / 
LLINK(P'), terminate (T ■< T'); if LLINK(P) / A = LLINK(P'), terminate 
(T y T'); otherwise set P -e- LLINK(P) , P' <- LLINK(P'), and go to R2. 

R4. [End of tree?] If P = HEAD (or, equivalently, if P' = HEAD'), terminate 
(T is equivalent to T'). 

R5. [Go to right.] If RTAG(P) = 1 = RTAG(P'), set P RLINK(P) , P' t- RLINK(P'), 
and go to R4. If RTAG(P) = 1 / RTAG(P'), terminate (T ^ T'). If RTAG(P) / 
1 = RTAG(P'), terminate ( T y T'). Otherwise, set P RLINK(P), P' <— 
RLINK(P'), and go to R2. | 

To prove the validity of this algorithm (and therefore to understand how it works), 
one may show by induction on the size of the tree To that the following statement is 
valid: Starting at step R2 with P and P' pointing to the roots of two nonempty right- 
threaded binary trees T 0 and To, the algorithm will terminate if T 0 and Tq are not 
equivalent, indicating whether To -< Tq or To y Tq; the algorithm will reach step R4 
if To and Tq are equivalent, with P and P' then pointing respectively to the successor 
nodes of To and Tq in symmetric order. 

28. Equivalent and similar. 

29. Prove by induction on the size of T that the following statement is valid: Starting 
at step C2 with P pointing to the root of a nonempty binary tree T and with Q pointing 
to a node that has empty left and right subtrees, the procedure will ultimately arrive 
at step C6 after setting INFO(Q) <— INFO(P) and attaching copies of the left and right 
subtrees of NODE(P) to NODE(Q) , and with P and Q pointing respectively to the preorder 
successor nodes of the trees T and NODE(Q) . 
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30. Assume that the pointer T in ( 2 ) is LLINK(HEAD) in ( 10 ). 

LI. [Initialize.] Set Q 4- HEAD, RLINK(Q) 4 - Q. 

L2. [Advance.] Set P 4— Q$. (See below.) 

L3. [Thread.] If RLINK(Q) = A, set RLINK(Q) 4 - P, RTAG(Q) 4 - 1; otherwise set 
RTAG(Q) 4- 0. If LLINK(P) = A, set LLINK(P) 4 - Q, LTAG(P) 4 - 1; otherwise 
set LTAG(P) 4 - 0. 

L4. [Done?] If P ^ HEAD, set q 4 — P and return to L2. | 

Step L2 of this algorithm implies the activation of an inorder traversal coroutine 
like Algorithm T, with the additional proviso that Algorithm T visits HEAD after it has 
fully traversed the tree. This notation is a convenient simplification in the description 
of tree algorithms, since we need not repeat the stack mechanisms of Algorithm T over 
and over again. Of course Algorithm S cannot be used during step L2, since the tree 
hasn’t been threaded yet. But the algorithm of exercise 21 can be used in step L2. 
and this provides us with a very pretty method that threads a tree without using any 
auxiliary stack. 

31. XI. Set P 4- HEAD. 

X2. Set Q e— P$ (using, say, Algorithm S, modified for a right-threaded tree). 

X3. If P / HEAD, set AVAIL 4 = P. 

X4. If Q yf HEAD, set P 4— q and go back to X2. 

X5. Set LLINK(HEAD) 4 - A. | 

Other solutions that decrease the length of the inner loop are clearly possible, although 
the order of the basic steps is somewhat critical. The stated procedure works because 
we never return a node to available storage until after Algorithm S has looked at both 
its LLINK and its RLINK; as observed in the text, each of these links is used precisely 
once during a complete tree traversal. 

32. RLINK (Q) -f— RLINK (P) , SUC(Q) 4 - SUC(P) , SUC(P) 4 — RLINK (P) 4 — Q, PRED(Q) 4 - P. 
PRED ( SUC ( Q ) ) 4 — q. 

33. Inserting NODE(Q) just to the left and below NODE(P) is quite simple: Set LLINKT(Q) 
LLINKT(P) , LLINK (P) 4 - Q, LTAG(P) 4 - 0, RLINK(Q) 4 - A. Insertion to the right is 

considerably harder, since it essentially requires Ending *Q, which is of comparable 
difficulty to finding Qa (see exercise 19); the node-moving technique discussed in ex- 
ercise 23 could perhaps be used. So general insertions are more difficult with this 
type of threading. But the insertions required by Algorithm C are not as difficult as 
insertions are in general, and in fact the copying process is slightly faster for this kind 
of threading: 

Cl. Set P 4 — HEAD, Q 4— U, go to C4. (The assumptions and philosophy of 
Algorithm C in the text are being used throughout.) 

C2. If RLINK (P ) / A, set R <= AVAIL, LLINK (R) 4- LLINK(Q), LTAG(R) 4 - 1, 
RLINK (R) 4 - A, RLINK(Q) 4 - LLINK(Q) 4 - R. 

C3. Set INF0(q) 4 - INFO (P) . 

C4. If LTAG(P) = 0, set R <= AVAIL, LLINK(R) 4 - LLINK(Q), LTAG(R) 4 - 1, 
RLINK(R) 4- A, LLINK(Q) 4 - R, LTAG(q) 4 - 0. 

C5. Set P 4 - LLINK (P) , q 4 - LLINK (q) . 

C6. If P / HEAD, go to C2. | 

The algorithm now seems almost too simple to be correct! 


2 . 3.2 


ANSWERS TO EXERCISES 571 


Algorithm C for threaded or right-threaded binary trees takes slightly longer due 
to the extra time to calculate P*, Q* in step C5. 

It would be possible to thread RLINKs in the usual way or to put HP in RLINK(P) , in 
conjunction with this copying method, by appropriately setting the values of RLINK(R) 
and RLINKT(Q) in steps C2 and C4. 

34. Al. Set Q <- P, and then repeatedly set Q RLINK(Q) zero or more times until 
RTAG(Q) = 1. 

A2. Set R <- RLINK(Q). If LLINK(R) = P, set LLINK(R) <- A. Otherwise set 
R «— LLINK(R) , then repeatedly set R <— RLINK(R) zero or more times until 
RLINK(R) = P; then finally set RLINKT(R) <- RLINKT(Q). (This step has 
removed NODE(P) and its subtrees from the original tree.) 

A3. Set RLINK(Q) <- HEAD, LLINK(HEAD) 4- P. | 

(The key to inventing and/or understanding this algorithm is the construction of good 
"before and after” diagrams.) 

36. No; see the answer to exercise 1.2.1 15(e). 

37. If LLINK(P) = RLINK(P) = A in the representation ( 2 ), let LINK(P) = A; otherwise 
let LINK(P) = Q where NODE(Q) corresponds to NODE(LLINK(P) ) and N0DE(Q + 1) 
to NODE(RLINK(P) ) . The condition LLINK(P) or RLINK(P) = A is represented by a 
sentinel in NODE(Q) or N0DE(Q + 1) respectively. This representation uses between n 
and 2n — 1 memory positions; under the stated assumptions, ( 2 ) would require 18 words 
of memory, compared to 11 in the present scheme. Insertion and deletion operations 
are approximately of equal efficiency in either representation. But this representation 
is not quite as versatile in combination with other structures. 

SECTION 2.3.2 

1. If B is empty, F(B) is an empty forest. Otherwise, F(B) consists of a tree T plus 
the forest F(right(B)), where root(T) = root(B) and subtrees(T) = F(left(B)). 

2. The number of zeros in the binary notation is the number of decimal points in the 
decimal notation; the exact formula for the correspondence is 

ai .a 2 .---. 0 fc <4 f ai oi“ 2_1 0 . . . 01 afc — 1 , 

where l a denotes a ones in a row. 

3. Sort the Dewey decimal notations for the nodes lexicographically (from left to 
right, as in a dictionary), placing a shorter sequence 01 . • • • .a*, in front of its extensions 
01 . • • • .as,. • • • ,a r for preorder, and behind its extensions for postorder. Thus, if we 
were sorting words instead of sequences of numbers, we would place the words cat, 
cataract in the usual dictionary order, to get preorder; we would reverse the order of 
initial subwords ( cataract , cat), to get postorder. These rules are readily proved by 
induction on the size of the tree. 

4. True, by induction on the number of nodes. 

5. (a) Inorder, (b) Postorder. It is interesting to formulate rigorous induction proofs 
of the equivalence of these traversal algorithms. 

6. We have preorder(T) = preorder(T'), and postorder(T) = inorder(T'), even if T 
has nodes with only one child. The remaining two orders are not in any simple relation; 
for example, the root of T comes at the end in one case and about in the middle in the 
other. 
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7. (a) Yes; (b) no; (c) no; (d) yes. Note that reverse preorder of a forest equals 
postorder of the left-right reversed forest (in the sense of mirror reflection). 

8. T^T' means that either info(root(T)) -C info(root(T')), or these info’s are equal 
and the following condition holds: Suppose the subtrees of root(T) are Ti, . . . , T n and 
the subtrees of root(T ) are T{, ... , T' n , , and let k > 0 be as large as possible such that 
Tj is equivalent to T- for 1 < j < k. Then either k = n or k < n and T k+1 < T' k+1 . 

9. The number of nonterminal nodes is one less than the number of right links that 
are A, in a nonempty forest, because the null right links correspond to the rightmost 
child of each nonterminal node, and also to the root of the rightmost tree in the forest. 
(This fact gives another proof of exercise 2.3.1-14, since the number of null left links 
is obviously equal to the number of terminal nodes.) 

10. The forests are similar if and only if n = n and d(u 3 ) = d(u'), for 1 < j < n; they 
are equivalent if and only if in addition info(wj) = info(u)), 1 < j < n. The proof is 
similar to the previous proof, by generalizing Lemma 2.3. IP; let f(u) = d(u) - 1. 

11 . 



12. If INFO (Ql) ^ 0: Set R 4 - C0PY(P1); then if TYPE(P2) = 0 and INF0(P2) ^ 2, 
set R 4- TREE (‘T, TREE ( INFO (P2) -1)); if TYPE(P2) / 0, set R 4- TREE(T',R> 
TREE( ‘'—".COPY (P2) , TREE ( 1 ) ) ) ; then set Ql -f- MULT (Ql .MULTICOPY (P2) ,R)) . 

If INFO(Q) / 0: Set Q <- TREE ( “‘ x ”, MULT (TREE( “ln”,C0PY (PI) ) ,Q) ,TREE( “t” 
COPY (PI) .COPY (P2) ) ) . 

Finally go to DIFF[4] . 

13. The following program implements Algorithm 2.3. 1C with rll = P, rI2 = Q. 


rI3 = 

R, 

and with appropriate changes to the initialization and termination conditions: 

064 


ST3 

6F(0 : 2) 

Save contents of rI3, rI2. 

065 


ST2 

7F(0 : 2) 

Cl. Initialize. 

066 


ENT2 

8F 

Start by creating N0DE(U) with 

067 


JMP 

IF 

RLINK(U) = A. 

068 

8H 

CON 

0 

Zero constant for initialization 

069 

4H 

LD1 

O.l(LLINK) 

Set P 4 - LLINK(P) = P*. 

070 

1H 

LD3 

AVAIL 

R AVAIL. 

071 


J3Z 

OVERFLOW 


072 


LDA 

0,3(LLINK) 


073 


STA 

AVAIL 


074 


ST3 

0,2(LLINK) 

LLINK(Q) 4 - R. 

075 


ENNA 

0,2 


076 


STA 

0,3(RLINKT) 

RLINK(R) 4 - Q, RTAG(R) 4 - 1. 


Ill 
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077 


INCA 

8B 

rA 4— L0C(init node) — Q. 

078 


ENT2 

0,3 

Set Q 4 — R = Q*. 

079 


JAZ 

C3 

To C3, the first time. 

080 

C2 

LDA 

0,1 

C2. Anything to right? 

081 


JAN 

C3 

Jump if RTAG(P) = 1. 

082 


LD3 

AVAIL 

R <= AVAIL. 

083 


J3Z 

OVERFLOW 


084 


LDA 

0,3(LLINK) 


085 


STA 

AVAIL 


086 


LDA 

0,2(RLINKT) 


087 


STA 

0,3(RLINKT) 

Set RLINKT(R) 4- RLINKT(Q). 

088 


ST3 

0,2(RLINKT) 

RLINK (Q) 4- R, RTAG(Q) 4 - 0. 

089 

C3 

LDA 

1,1 

C3. Copy INFO. 

090 


STA 

1,2 

INFO field copied. 

091 


LDA 

0,1 (TYPE) 


092 


STA 

0,2 (TYPE) 

TYPE Held copied. 

093 

C4 

LDA 

0,1(LLINK) 

C4. Anything to left? 

094 


JANZ 

4B 

Jump if LLINK(P) ± A. 

095 


STZ 

0,2(LLINK) 

LLINK(Q) 4 - A. 

096 

C5 

LD2N 

0,2(RLINKT) 

C5. Advance. 0 4 RLINKT(O) . 

097 


LD1 

0 , 1 (RLINK) 

P 4 - RLINK (P). 

098 


J2P 

C5 

Jump if RTAG(Q) was I. 

099 


ENN2 

0,2 

Q -Q. 

100 

C6 

J2NZ 

C2 

C6. Test if complete. 

101 


LD1 

8B(LLINK) 

rll 4— location of first node created. 

102 

6H 

ENT3 

* 

Restore index registers. 

103 

7H 

ENT2 

* 

1 


14. Let a be the number of nonterminal (operator) nodes copied. The number of 
executions of the various lines in the previous program is as follows: 064-067, 1; 069, a; 
070-079, a + 1; 080-081, n - 1; 082-088, n - 1 - a; 089-094, n; 095, n - a; 096-098, 
n + 1; 099-100, n — a; 101-103, 1. The total time is (36n + 22 )u; we use about 20% of 
the time to get available nodes, 40% to traverse, and 40% to copy the INFO and LINK 
information. 


15. Comments are left to the reader. 


218 

DIV 

LDA 

1,6 

231 

ENTA 

UP ARROW 

219 


JAZ 

IF 

232 

1H ENTX 

* 

220 


JMP 

C0PYP2 

233 

JMP 

TREE2 

221 


ENTA 

SLASH 

234 

ST1 

IF (0:2) 

222 


ENTX 

0,6 

235 

JMP 

C0PYP1 

223 


JMP 

TREE2 

236 

ENTA 

0,1 

224 


ENT6 

0,1 

237 

ENT1 

0,5 

225 

1H 

LDA 

1,5 

238 

JMP 

MULT 

226 


JAZ 

SUB 

239 

ENTX 

0,1 

227 


JMP 

C0PYP2 

240 

1H ENT1 

* 

228 


ST1 

IF (0 : 2) 

241 

ENTA 

SLASH 

229 


ENTA 

C0N2 

242 

JMP 

TREE2 

230 


JMP 

TREE0 

243 

ENT5 

0,1 





244 

JMP 

SUB | 
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16. Comments are left to the reader. 


245 

PWR 

LDA 

1,6 

263 


JMP 

TREEO 

281 


ENTA 

LOG 

246 


JAZ 

4F 

264 

1H 

ENTX 

* 

282 


JMP 

TREE1 

247 


JMP 

C0PYP1 

265 


ENTA 

MINUS 

283 


ENTA 

0,1 

248 


ST1 

R(0 : 2) 

266 


JMP 

TREE2 

284 


ENT1 

0,5 

249 


LDA 

0,3 (TYPE) 

267 

5H 

LDX 

R(0 : 2) 

285 


JMP 

MULT 

250 


JANZ 

2F 

268 


ENTA 

UP ARROW 

286 


ST1 

IF (0 : 2) 

251 


LDA 

1,3 

269 


JMP 

TREE 2 

287 


JMP 

C0PYP1 

252 


DECA 

2 

270 


ST1 

R(0 : 2) 

288 


ST1 

2F(0 : 2) 

253 


JAZ 

3F 

271 

3H 

JMP 

C0PYP2 

289 


JMP 

C0PYP2 

254 


INCA 

1 

272 


ENTA 

0,1 

290 

2H 

ENTX 

* 

255 


STA 

C0N0+1 

273 

R 

ENT1 

* 

291 


ENTA 

UP ARROW 

256 


ENTA 

C0N0 

274 


JMP 

MULT 

292 


JMP 

TREE2 

257 


JMP 

TREEO 

275 


ENTA 

0,6 

293 

1H 

ENTX 

* 

258 


STZ 

C0N0+1 

276 


JMP 

MULT 

294 


ENTA 

TIMES 

259 


JMP 

5F 

277 


ENT 6 

0,1 

295 


JMP 

TREE2 

260 

2H 

JMP 

C0PYP2 

278 

4H 

LDA 

1,5 

296 


ENT5 

0,1 

261 


ST1 

IF (0 : 2) 

279 


JAZ 

ADD 

297 


JMP 

ADD | 

262 


ENTA 

CONI 

280 


JMP 

C0PYP1 






17. References to early work on such problems can be found in a survey article by 
J. Sammet, CACM 9 (I960), 555-569. 

18. First set LLINK [j] 4— RLINK [j] 4— j for all j, so that each node is in a circular 
list of length 1. Then for j - n, n - 1, . . . , 1 (in this order), if PARENT [ j] = 0 set 
r j i otherwise insert the circular list starting with j into the circular list starting with 
PARENT [j] as follows: k 4- PARENT [j], l 4- RLINK [fc] , i 4- LLINK [ ;] . LLINK [j] 4- k, 
RLINK [A:] 4— j, LLINK [/] 4— i, RLINK [j] 4— l. This works because (a) each nonroot 
node is always preceded by its parent or by a descendant of its parent; (b) nodes of 
each family appear in their parent’s list, in order of location; (c) preorder is the unique 
order satisfying (a) and (b). 

20. If u is an ancestor of v, it is immediate by induction that u precedes v in preorder 
and follows v in postorder. Conversely, suppose u precedes v in preorder and follows 
v in postorder; we must show that u is an ancestor of v. This is clear if u is the root 
of the first tree. If u is another node of the first tree, v must be also, since u follows v 
in postorder; so induction applies. Similarly if u is not in the first tree, v must not be 
either, since u precedes v in preorder. (This exercise also follows easily from the result 
of exercise 3. It gives us a quick test for ancestorhood, if we know each node’s position 
in preorder and postorder.) 

21. If NODE(P) is a binary operator, pointers to its two operands are PI = LLINK (P) 
and P2 = RLINK (PI) = sP. Algorithm D makes use of the fact that P2$ = P, so 
that, RLINK(Pl) may be changed to Ql, a pointer to the derivative of NODE(Pl); then 
RLINK(Pl) is reset later in step D3. For ternary operations, we would have, say, 
PI = LLINK (P), P2 = RLINK(Pl) , P3 = RLINK (P2) = $P, so it is difficult to generalize 
the binary trick. After computing the derivative Ql, we could set RLINK(Pl) 4 - Ql tem- 
porarily, and then after computing the next derivative Q2 we could set RLINK (Q2) 4 — Ql 
and RLINK (P2) 4 Q2 and reset RLINK(Pl) 4— P2. But this is certainly inelegant, and it 
becomes progressively more so as the degree of the operator becomes higher. Therefore 
the device of temporarily changing RLINK (PI) in Algorithm D is definitely a trick , 


2 . 3.3 


ANSWERS TO EXERCISES 575 


not a technique. A more aesthetic way to control a differentiation process, because it 
generalizes to operators of higher degree and does not rely on isolated tricks, can be 
based on Algorithm 2.3.3F; see exercise 2. 3. 3-3. 

22. From the definition it follows immediately that the relation is transitive; that is, 
if T C T' and T' C T" then T C T". (In fact the relation is easily seen to be a partial 
ordering.) If we let / be the function taking nodes into themselves, clearly l(T) C T 
and r(T) C T. Therefore if T C Z(T') or T C r(T') we must have T C T'. 

Suppose fi and f r are functions that respectively show l(T) C l(T') and r(T) C 
r(T'). Let f(u) = fi(u) if u is in Z(T), f(u) = root(T') if u is root(T), otherwise 
f(u) — fr{u). Now it follows easily that / shows T C T'; for example, if we let r'(T) 
denote r(T) \ root (T) we have preorder(T) = root(T) preorder(Z(T)) preorder(r'(T)); 
preorder(T') = /(root(T)) preorder(Z(T')) preorder(r'(T')). 

The converse does not hold: Consider the subtrees with roots b and b 1 in Fig. 25. 

SECTION 2.3.3 

1. Yes, we can reconstruct them just as (3) is deduced from (4), but interchanging 
LTAG and RTAG, LLINK and RLINK, and using a queue instead of a stack. 

2. Make the following changes in Algorithm F: Step FI, change to “last node of the 

forest in preorder.” Step F2, change to u f(xi f(xd)” in two 

places. Step F4, “If P is the first node in preorder, terminate the algorithm. (Then the 
stack contains /(root(Tj)), . . . , /(root(T m )), from top to bottom, where T\, . . . ,T m are 
the trees of the given forest, from left to right.) Otherwise set P to its predecessor in 
preorder (P 4— P — c in the given representation), and return to F2.” 

3. In step Dl, also set S 4— A. (S is a link variable that links to the top of the 
stack.) Step D2 becomes, for example, “If NODE(P) denotes a unary operator, set 
Q 4— S, S 4— RLINK(Q), PI 4— LLINK (P); if it denotes a binary operator, set Q 4 — S, 
Q1 4- RLINK(Q), S 4- RLINK(Ql), PI 4- LLINK (P), P2 4- RLINK(Pl). Then perform 
DIFF[TYPE(P)] .” Step D3 becomes “Set RLINK(Q) 4— S, S 4 — Q.” Step D4 becomes 
“Set P 4— P$.” The operation LLINK(DY) +- Q may be avoided in step D5 if we assume 
that S = LLINK (DY). This technique clearly generalizes to ternary and higher-order 
operators. 

4. A representation like (10) takes n — m LLINKs and n + (n — m) RLINKs. The 
difference in total number of links is n — 2m. between the two forms of representation. 
Arrangement (10) is superior when the LLINK and INFO fields require about the same 
amount of space in a node and when m is rather large, namely when the nonterminal 
nodes have rather large degrees. 

5. It would certainly be silly to include threaded RLINKs, since an RLINK thread just 
points to PARENT anyway. Threaded LLINKs as in 2.3.2 ( 4 ) would be useful if it is 
necessary to move leftward in the tree, for example if we wanted to traverse a tree in 
reverse postorder, or in family order; but these operations are not significantly harder 
without threaded LLINKs unless the nodes tend to have very high degrees. 

6. LI. Set P <- FIRST, FIRST <- A. 

L2. If P = A, terminate. Otherwise set Q RLINK (P) . 

L3. If PARENT (P) = A, set RLINK (P) <- FIRST, FIRST 4- P; otherwise set R f- 
PARENT(P) , RLINK(P) <- LCHILD(R) , LCHILD(R) 4- P. 

L4. Set P 4— Q and return to L2. | 
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7. {1, 5}{2, 3, 4, 7}{6, 8, 9}. 

8. Perform step E3 of Algorithm E, then test if j = k. 

9. PARENT [fc] : 502208228 

k : 123456789 



10. One idea is to set PARENT of each root node to the negative of the number of 
nodes in its tree (these values being easily kept up to date); then if | PARENT [j] J > 

) PARENT [fc] | in step E4, the roles of j and k are interchanged. This technique (due to 
M. D. Mcllroy) ensures that each operation takes O(logn) steps. 

For still more speed, we can use the following suggestion due to Alan Tritter: In 
step E4, set PARENT [x] 4- k for all values x / k that were encountered in step E3. 
This makes an extra pass up the trees, but it collapses them so that future searches 
are faster. (See Section 7.4.1.) 

11. It suffices to define the transformation that is done for each input (P, j, Q, k)\ 

Tl. If PARENT(P) A, set j 4— j + DELTA(P), P 4— PARENT(P), and repeat this 
step. 

T2. If PARENT (Q) A, set k 4 k — DELTA (Q), Q 4— PARENT (Q), and repeat this 
step. 

T3. If P = Q, check that j = k (otherwise the input erroneously contains con- 
tradictory equivalences). If P / q, set DELTA (Q) <- j - k, PARENT (Q) 4 - P. 
LBD(P) 4- min(LBD(P), LBD(q) + DELTA ( Q )) , and UBD(P) 4 - max(UBD(P). 
UBD(Q) +DELTA(Q)). | 

Note: It is possible to allow the ARRAY X[<:«] declarations to occur intermixed with 
equivalences, or to allow assignment of certain addresses of variables before others are 
equivalenced to them, etc.., under suitable conditions that are not difficult to under- 
stand. For further development of this algorithm, see CACM 7 (1964), 301-303, 506. 

12. (a) Yes. (If this condition is not required, it would be possible to avoid the loops 
on S that appear in steps A2 and A9.) (b) Yes. 

13. The crucial fact is that the UP chain leading upward from P always mentions the 
same variables and the same exponents for these variables as the UP chain leading 
upward from q, except that the latter chain may include additional steps for variables 
with exponent zero. (This condition holds throughout most of the algorithm, except 
during the execution of steps A9 and A10.) Now we get to step A8 either from A3 or 
from A 10, and in each case it was verified that EXP(q) / 0. Therefore EXP(P) ^ 0. 
and in particular it follows that P / A, q ^ A, UP(P) ^ A, UP(q) ^ A; the result 
stated in the exercise now follows. Thus the proof depends on showing that the UP 
chain condition stated above is preserved by the actions of the algorithm. 

16. We prove (by induction on the number of nodes in a single tree T) that if P is 
a pointer to T, and if the stack is initially empty, steps F2 through F4 will end with 
the single value /(root(T)) on the stack. This is true for n = 1. If n > 1, there are 
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0 < d = DEGREE(root(T)) subtrees Ti, . . . , T,i\ by induction and the nature of a stack, 
and since postorder consists of Ti, . . . ,T d followed by root (T), the algorithm computes 
/(Ti), . . . ,/(T d ), and then /(root(T)), as desired. The validity of Algorithm F for 
forests follows. 

17. Gl. Set the stack empty, and let P point to the root of the tree (the last node in 

postorder). Evaluate /(NODE (P) ). 

G2. Push DEGREE(P) copies of /(NODE(P)) onto the stack. 

G3. If P is the first node in postorder, terminate the algorithm. Otherwise set P 
to its predecessor in postorder (this would be simply P P — c in (9)). 

G4. Evaluate /(NODE(P)) using the value at the top of the stack, which is equal 
to /(NODE (PARENT (P) ) ). Pop this value off the stack, and return to G2. | 

Note: An algorithm analogous to this one can be based on preorder instead of postorder 
as in exercise 2. In fact, family order or level order could be used; in the latter case we 
would use a queue instead of a stack. 

18. The INF01 and RLINK tables, together with the suggestion for computing LTAG in 
the text, give us the equivalent of a binary tree represented in the usual manner. The 
idea is to traverse this tree in postorder, counting degrees as we go: 

PI. Let R, D, and I be stacks that are initially empty; then set R <= n + 1, D <= 0, 

j <— 0, k <— 0. 

P2. If top(R) > j + 1, go to P5. (If an LTAG field were present, we could have 
tested LTAG [/] = 0 instead of top(R) > j + 1.) 

P3. If I is empty, terminate the algorithm; otherwise set i <= I, k <— k + 1, 
INF02 [fc] INF01 [j] , DEGREE [fc] 4= D. 

P4. If RLINK [j] = 0, go to P3; otherwise delete the top of R (which will equal 
RLINK [i]). 

P5. Set top(D) <r- top(D) + 1, j <r— j + 1, I <= j, D <*= 0, and if RLINK [/] 7^ 0 set 
R <= RLINK [/] . Go to P2. | 

19. (a) This property is equivalent to saying that SCOPE links do not cross each other. 

(b) The first tree of the forest contains di + 1 elements, and we can proceed by induction. 

(c) The condition of (a) is preserved when we take minima. 

Notes: By exercise 2.3.2—20, it follows that did? . . . d n can also be interpreted in 
terms of inversions: If the fcth node in postorder is the p^th node in preorder, then d k 
is the number of elements > k that appear to the left of k in P 1 P 2 . . .p„. 

A similar scheme, in which we list the number of descendants of each node in 
postorder of the forest, leads to sequences of numbers C1C2 . . .c„ characterized by the 
properties (i) 0 < c k < k and (ii) k > j > k — Ck implies j — Cj > k — Ck- Algorithms 
based on such sequences have been investigated by J. M. Pallo, Comp. J. 29 (1986), 
171-175. Notice that c k is the size of the left subtree of the fcth node in symmetric 
order of the corresponding binary tree. We can also interpret dk as the size of the right 
subtree of the kth node in symmetric order of a suitable binary tree, namely the binary 
tree that corresponds to the given forest by the dual method of exercise 2. 3. 2-5. 

The relation d k < d' k for 1 < k < n defines an interesting lattice ordering of forests 
and binary trees, first introduced in another way by D. Tamari [These (Paris, 1951)]; 
see exercise 6.2.3-32. 
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SECTION 2.3.4. 1 

1. (B, A, C, D, B), ( B,A,C,D,E,B ), (B,D,C,A,B), (B,D,E,B), ( B,E,D,B ), 
(B,E,D,C,A,B). 

2. Let (Vb, Vi, . . . , V n ) be a path of smallest possible length from V to V'. If now 
Vj = Vk for some j < k, then (Vo, . . . , Vj, I4+i, . . . , V n ) would be a shorter path. 

3. (The fundamental path traverses e 3 and e 4 once, but cycle Cg traverses them -1 
times, giving a net total of zero.) Traverse the following edges: ei, e 2 , e6, ej, eg, eio, 

en, ei 2 , ei4. 

4. If not, let G" be the subgraph of G' obtained by deleting each edge ej for which 
Ej = 0. Then G" is a finite graph that has no cycles and at least one edge, so by 
the proof of Theorem A there is at least one vertex, V, that is adjacent to exactly one 
other vertex, V'. Let ej be the edge joining V to V 1 ; then Kirchhoff’s equation (i) at 
vertex V is Ej = 0, contradicting the definition of G". 

5. A = 1 + E s , B = 1 + Es — Eg, C = 1 + Eg, D = 1 + Eg — Eg, E = 1 + En — Eg i, 
F = 1 + F” 3 + En — E 21 , G = 1 + E" 3 , H = En — E 2 \, J = En, K = E" 9 + Ego, 
L = En + E ", 9 + Eg 0 — Eg 1, P = En + Ego — Eg 1, Q = Ego, R = En — Egi, S = Egg. 
Note: In this case it is also possible to solve for Eg, Eg,..., Egg in terms of A, B, . . . , 5; 
hence there are nine independent solutions, explaining why we eliminated six variables 
in Eq. 1.3.3-(8). 

6. (The following solution is based on the idea that we may print out each edge that 
does not make a cycle with the preceding edges.) Use Algorithm 2.3.3E, with each pair 
( ai,bi ) representing a t = bi in the notation of that algorithm. The only change is to 
print ( a,,bi ) if j / k in step E4. 

To show that this algorithm is valid, we must prove that (a) the algorithm prints 
out no edges that form a cycle, and (b) if G contains at least one free subtree, the 
algorithm prints out n - 1 edges. Deflne j = k if there exists a path from Vj to 14 
or if j = k. This is clearly an equivalence relation, and moreover j = k if and only 
if this relation can be deduced from the equivalences ai = 61, . . . , o m = 6 m . Now 
(a) holds because the algorithm prints out no edges that form a cycle with previously 
printed edges; (b) is true because PARENT [fc] = 0 for precisely one k if all vertices are 
equivalent. 

A more efficient algorithm can, however, be based on depth-first search; see 
Algorithm 2.3.5A and Section 7.4.1. 

7 . Fundamental cycles: Co = eo + ei + e 4 + e 9 (fundamental path is ei + e 4 + e 9 ); 
Cg = eg + e 3 + e 2 ; Ce = e 6 — e 2 + e 4 ; C7 = e7 — e 4 — e 3 \ C s = eg — e 9 — e 4 — e 3 . 
Therefore we find E x - 1, Eg = Eg - Eg, E 3 = Eg - E 7 - Eg, E 4 = 1 + Eg - E 7 - Eg, 
Eg — 1 — Eg. 

8. Each step in the reduction process combines two arrows e; and ej that start at 
the same box, and it suffices to prove that such steps can be reversed. Thus we are 
given the value of e, + ej after combination, and we must assign consistent values to ej 
and ej before the combination. There are three essentially different situations: 

Case 1 Case 2 Case 3 


Before 
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After 


ei+e 




a 

p 



p 1 

p" 


p 




Here A, B, and C stand for vertices or supervertices, and the a’s and p’s stand for the 
other given flows besides a + Vj ; these flows may each be distributed among several 
edges, although only one is shown. In Case 1 (ei and e 3 lead to the same box), we may 
choose e* arbitrarily, then e 3 4— (a + e 3 ) — ej. In Case 2 (ei and e 3 lead to different 
boxes), we must set a «- P' - a, ej «- P" - a”. In Case 3 (a is a loop but e } is 
not), we must set e 3 P' — a', e 3 <— (a + ej) — ej. In each case we have reversed the 
combination step as desired. 

The result of this exercise essentially proves that the number of fundamental cycles 
in the reduced flow chart is the minimum number of vertex flows that must be measured 
to determine all the others. In the given example, the reduced flow chart reveals that 
only three vertex flows (e.g., a, c, d ) need to be measured, while the original chart 
of exercise 7 has four independent edge flows. We save one measurement every time 
Case 1 occurs during the reduction. 

A similar reduction procedure could be based on combining the arrows flowing into 
a given box, instead of those flowing out. It can be shown that this would yield the 
same reduced flow chart, except that the supervertices would contain different names. 

The construction in this exercise is based on ideas due to Armen Nahapetian and 
F. Stevenson. For further comments, see A. Nahapetian, Acta Informatics 3 (1973), 
37-41; D. E. Knuth and F. Stevenson, BIT 13 (1973), 313-322. 

9. Each edge from a vertex to itself becomes a “fundamental cycle” all by itself. If 
there are k + 1 edges e, e', . . . , e (fc) between vertices V and V\ make k fundamental 
cycles e ± e, . . . , e* fc) ± e (choosing + or — according as the edges go in the opposite 
or the same direction), and then proceed as if only edge e were present. 

Actually this situation would be much simpler conceptually if we had defined a 
graph in such a way that multiple edges are allowed between vertices, and edges are 
allowed from a vertex to itself; paths and cycles would be defined in terms of edges 
instead of vertices. Such a definition is, in fact, made for directed graphs in Section 
2.3.4. 2. 

10 . If the terminals have all been connected together, the corresponding graph must 
be connected in the technical sense. A minimum number of wires will involve no cycles, 
so we must have a free tree. By Theorem A, a free tree contains n — 1 wires, and a 
graph with n vertices and n — 1 edges is a free tree if and only if it is connected. 

11. It is sufficient to prove that when n > 1 and c(n — 1, n) is the minimum of the 
c(i,n), there exists at least one minimum cost tree in which T n - 1 is wired to T n . (For, 
any minimum cost tree with n > 1 terminals and with T n _i wired to T n must also be a 
minimum cost tree with n — 1 terminals if we regard T n _i and T n as “common,” using 
the convention stated in the algorithm.) 

To prove the statement above, suppose we have a minimum cost tree in which 
T n _i is not wired to T„. If we add the wire T n _i — T n we obtain a cycle, and any 
of the other wires in that cycle may be removed; removing the other wire touching T n 
gives us another tree, whose total cost is not greater than the original, and T n -\ — T n 
appears in that tree. 
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12. Keep two auxiliary tables, a(i) and b(i), for 1 < i < n, representing the fact that 
the cheapest connection from T) to a chosen terminal is to T b(i) , and its cost is o(i); 
initially a(i) = c(i, n) and b(i) = n. Then do the following operation n — 1 times: Find 
i such that a(i) = min 1 < J< „ a(j); connect T, to T 6(i) ; for 1 < j < n if c(i,j) < a{j) set 
a (j) t— c(i,j) and b(j) i; and set a(i) <— oo. Here c(i,j ) means c(j,i) when j < i. 

(It is somewhat more efficient to avoid the use of oo, keeping instead a one- 
way linked list of those j that have not yet been chosen. With or without this 
straightforward improvement, the algorithm takes 0(n 2 ) operations.) See also E. W. 
Dijkstra, Proc. Nederl. Akad. Wetensch. A63 (1960), 196-199; D. E. Knuth, The 
Stanford GraphBase (New York: ACM Press, 1994), 460-497. Significantly better 
algorithms to find a minimum-cost spanning tree are discussed in Section 7.5.4. 

13. If there is no path from V, to V), for some i ^ j, then no product of the 
transpositions will move i to j. So if all permutations are generated, the graph must be 
connected. Conversely if it is connected, remove edges if necessary until we have a free 
tree. Then renumber the vertices so that V n is adjacent to only one other vertex, namely 
V n -i- (See the proof of Theorem A.) Now the transpositions other than (n— 1 n) form 
a free tree with n - 1 vertices; so by induction if 7r is any permutation of ( 1 , 2, . . . , n) 
that leaves n fixed, n can be written as a product of those transpositions. If n moves 
n to j then n(j n— l)(n— 1 n) = p fixes n\ hence 7r = p(n — 1 n)(j n— 1) can be written 
as a product of the given transpositions. 


SECTION 2. 3.4. 2 

1. Let (ei, . . . , e„) be an oriented path of smallest possible length from V to V'. If 
now init(e J ) = init(et) for j < k, (ei, . . . , e^-i, e*, . . . , e„) would be a shorter path; a 
similar argument applies if fin(e J -) = fin(e fc ) for j < k. Hence (ei, . . . ,e„) is simple. 

2. Those cycles in which all signs are the same: C 0 , C a , C " 3 , Ci 7 , C" 9 , C 20 . 

3. For example, use three vertices A, B, C, with arcs from A to B and A to C. 

4. If there are no oriented cycles, Algorithm 2.2.3T topologically sorts G. If there 
is an oriented cycle, topological sorting is clearly impossible. (Depending on how this 
exercise is interpreted, oriented cycles of length 1 could be excluded from consideration.) 

5. Let k be the smallest integer such that fin(e*) = init(e 3 ) for some j < k. Then 
(ej , . . . , ek) is an oriented cycle. 

6. False (on a technicality), just because there may be several different arcs from one 
vertex to another. 


7. True for finite directed graphs: If we start at any vertex V and follow the only 
possible oriented path, we never encounter any vertex twice, so we must eventually 
reach the vertex R (the only vertex with no successor). For infinite directed graphs the 
result is obviously false since we might have vertices R, Vi, V 2 , V 3 , . . . and arcs from V, 
to Vj+i for j > 1. 
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10. Gl. Set k <- P[j], P[j } <r- 0 . 

G2. If k = 0 , stop; otherwise set m P[k], P[k ] < - j, j k, k i- m, and repeat 
step G 2 . | 

11. This algorithm combines Algorithm 2 . 3 . 3 E with the method of the preceding 
exercise, so that all oriented trees have arcs that correspond to actual arcs in the 
directed graph; S[j ] is an auxiliary table that tells whether an arc goes from j to P[j] 
(S[j] = + 1 ) or from P[j] to j (S[j] = -1). Initially jP[ 1 ] = • • • = P[n ] = 0 . The 
following steps may be used to process each arc (a, b): 

Cl. Set j <- a, k <- P[j], P[j] 0 , s <- S[j]. 

C2. If k = 0 , go to C 3 ; otherwise set m <- P[k], t <- S[fc], P[fc] <- j , S[k] i s, 

s t, j <— k, k «— m, and repeat step C 2 . 

C3. (Now a appears as the root of its tree.) Set j ■<— b, and then if P[j] ^ 0 
repeatedly set j <— P[j] until P[j] = 0 . 

C 4 . If j = a, go to C 5 ; otherwise set P[a] <— b, S[a] i hi, print (a,b) as an arc 

belonging to the free subtree, and terminate. 

C 5 . Print “CYCLE” followed by “(a, 6)”. 

C6. If P[b] = 0 terminate. Otherwise if S[b] = + 1 , print “+(&, P[b])”, else print 
(P[6], b)”; set b <— P[b] and repeat step C6. | 

Note: This algorithm will take at most 0 (m log n) steps if we incorporate the suggestion 
of Mcllroy in answer 2 . 3 . 3 - 10 . But there is a much better solution that needs only O(m) 
steps: Use depth-first search to construct a “palm tree,” with one fundamental cycle 
for each “frond” [R. E. Tarjan, SICOMP 1 ( 1972 ), 146 - 150 ]. 

12. It equals the in-degree; the out-degree of each vertex can be only 0 or 1 . 

13. Define a sequence of oriented subtrees of G as follows: Go is the vertex R alone. 
Gk + 1 is Gy- . plus any vertex V of G that is not in Gk but for which there is an arc from 
V to V where V' is in Gk, plus one such arc e[V] for each such vertex. It is immediate 
by induction that Gk is an oriented tree for all k > 0, and that if there is an oriented 
path of length k from V' to R in G then V is in G k - Therefore Goo, the set of all V 
and e[V] in any of the G k , is the desired oriented subtree of G. 

14. (e 12 , e 2 o, e 0 o, e' 01 , e 10 , e 01 , e' 12 , e 22 , e 21 ), (e 12 , e 20 , e 0 o, eo lt e' 12 , e 22 , e 21 , e 10 , e 01 ), 

( e 12> e 20> e 01, e 10, e 00, e 01, e 12, e 22, e 2l), ( e 12, e 20, e 01, e 12, e 22, e 21, e 10> e 00> e 0l), 

( e 12, e 22, e 20, e 00, e 01, e 10, e 01, e 12, e 2l), ( e 12, e 22, e 20, e 00, e 01> e 12, e 21 , e 10, e 0l), 

( e i2, e 2 2, e20, e oi, eio, e 00 , e 01 , e' 12 , e 21 ), (ej 2 , e 22 , e 20 , e' 01 , e[ 2 , e 21 ,e 10 , e 00 , e 01 ), 

in lexicographic order; the eight possibilities come from the independent choices of 
which of e 0 o or e' 01 , e 10 or e' 12 , e 20 or e 22 , should precede the other. 

15 . True for finite graphs: If it is connected and balanced and has more than one 
vertex, it has an Eulerian trail that touches all the vertices. (But false in general.) 

16. Consider the directed graph G with vertices Vi, . . . , V 13 and with an arc from Vj 
to 14 for each k in pile j; this graph is balanced. Winning the game is equivalent to 
tracing out an Eulerian trail in G, because the game ends when the fourth arc to V13 
is encountered (namely, when the fourth king turns up). Now if the game is won, the 
stated digraph is an oriented subtree by Lemma E. Conversely if the stated digraph is 
an oriented tree, the game is won by Theorem D. 

17. This answer can be obtained, as the author first obtained it, by laborious enu- 
meration of oriented trees of special types and the application of generating functions, 
etc., based on the methods of Section 2 . 3 . 4 . 4 . But such a simple answer deserves a 
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simple, direct proof, and indeed there is one. Define an order for turning up all cards 
of the deck, as follows: Obey the rules of the game until getting stuck, then “cheat” by 
turning up the first available card (find the first pile that is not empty, going clockwise 
from pile 1) and continue as before, until eventually all cards have been turned up. 
The cards in the order of turning up are in completely random order (since the value 
of a card need not be specified until after it is turned up). So the problem is just 
to calculate the probability that in a randomly shuffled deck the last card is a king. 
More generally the probability that k cards are still face down when the game is over 
is the probability that the last king in a random shuffle is followed by k cards, namely 
4! ( 3 ” ) Hi • Hence a person playing this game without cheating will turn up an average 
of exactly 42.4 cards per game. Note: Similarly, it can be shown that the probability 
that the player will have to “cheat” k times in the process described above is exactly 
given by the Stirling number [ fc +J/13!. (See Eq. 1.2.10-(g) and exercise 1.2.10-7; the 
case of a more general card deck is considered in exercise 1.2.10-18.) 

18. (a) If there is a cycle (Vo, Vi, ... , Vj,), where necessarily 3 < k < n, the sum of the 
k rows of A corresponding to the k edges of this cycle, with appropriate signs, is a row 
of zeros; so if G is not a free tree the determinant of Aq is zero. 

But if G is a free tree we may regard it as an ordered tree with root V 0 , and we 
can rearrange the rows and columns of Ao so that columns are in preorder and so that 
the fcth row corresponds to the edge from the fcth vertex (column) to its parent. Then 
the matrix is triangular with il’s on the diagonal, so the determinant is ±1. 

(b) By the Binet-Cauchy formula (exercise 1.2.3-46) we have 

det A^A 0 = (det Aq ...j n ) 2 

l<il < • • * <in 

where A{ 1 ...i n represents a matrix consisting of rows i \ , . . . , i n of Aq (thus corresponding 
to a choice of n edges of G). The result now follows from (a). 

[See S. Okada and R. Onodera, Bull. Yamagata Univ. 2 (1952), 89-117.] 

19. (a) The conditions a 0 o = 0 and ajj = 1 are just conditions (a), (b) of the definition 
of oriented tree. If G is not an oriented tree there is an oriented cycle (by exercise 7), 
and the rows of A 0 corresponding to the vertices in this oriented cycle will sum to a 
row of zeros; hence det A 0 = 0. If G is an oriented tree, assign an arbitrary order to the 
children of each family and regard G as an ordered tree. Now permute rows and columns 
of Ao until they correspond to preorder of the vertices. Since the same permutation 
has been applied to the rows as to the columns, the determinant is unchanged; and the 
resulting matrix is triangular with +1 in every diagonal position. 

(b) We may assume that aoj = 0 for all j, since no arc emanating from Vo can 
participate in an oriented subtree. We may also assume that a 33 > 0 for all j > 1 since 
otherwise the whole jth row is zero and there obviously are no oriented subtrees. Now 
use induction on the number of arcs: If a 33 > 1 let e be some arc leading from V 3 ; let 
Bo be a matrix like Ao but with arc e deleted, and let Co be the matrix like Aq but 
with all arcs except e that lead from V 3 deleted. Example: If A 0 = ( _ 3 ~ 2 ), j = 1, 
and e is an arc from Vj to Vo, then Bo = ( _ 2 2 )> Co = ( ° )• In general we have 

det A 0 = det Bo + detC 0 , since the matrices agree in all rows except row j, and A 0 is 
the sum of Bo and Co in that row. Moreover, the number of oriented subtrees of G 
is the number of subtrees that do not use e (namely, det_B 0 , by induction) plus the 
number that do use e (namely, detCo). 

Notes. The matrix A is often called the Laplacian of the graph, by analogy with a 
similar concept in the theory of partial differential equations. If we delete any set 5 of 
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rows from the matrix A , and the same set of columns, the determinant of the resulting 
matrix is the number of oriented forests whose roots are the vertices {14 | k £ Sj and 
whose arcs belong to the given digraph. The matrix tree theorem for oriented trees 
was stated without proof by J. J. Sylvester in 1857 (see exercise 28), then forgotten for 
many years until it was independently rediscovered by W. T. Tutte [Proc. Cambridge 
Phil. Soc. 44 (1948), 463-482, §3], The first published proof in the special case of 
undirected graphs, when the matrix A is symmetric, was given by C. W. Borchardt 
[Crelle 57 (1860), 111-121], Several authors have ascribed the theorem to Kirchhoff, 
but Kirchhoff proved a quite different (though related) result. 

20. Using exercise 18 we find B = AlA 0 . Or, using exercise 19, B is the matrix A 0 for 
the directed graph G' with two arcs (one in each direction) in place of each edge of G; 
each free subtree of G corresponds uniquely to an oriented subtree of G' with root Vo, 
since the directions of the arcs are determined by the choice of root. 

21. Construct the matrices A and A* as in exercise 19. For the example graphs G 
and G* in Figs. 36 and 37, 


o 




[00] 

[10] 

[20] 

[01] 

[01] 

[21] 

[12] 

[12] 

[22] 




[00] 

( 2 

0 

0 

-1 

-1 

0 

0 

0 

°\ 




[10] 

-1 

3 

0 

-1 

-1 

0 

0 

0 

0 




[20] 

-1 

0 

3 

-1 

-1 

0 

0 

0 

0 

2 -2 

°\ 


[01] 

0 

-1 

0 

3 

0 

0 

-1 

-1 

0 

1 3 

— 2 

, A * 

= [oi] 

0 

-1 

0 

0 

3 

0 

-1 

-1 

0 

1 -1 

2/ 


[21] 

0 

-1 

0 

0 

0 

3 

-1 

-1 

0 




[12] 

0 

0 

-1 

0 

0 

-1 

3 

0 

-1 




[12] 

0 

0 

-1 

0 

0 

-1 

0 

3 

-1 




[22] 

^ 0 

0 

-1 

0 

0 

-1 

0 

0 

2/ 


Add the indeterminate A to every diagonal element of A and A*. If t(G) and f(G*) are 
the numbers of oriented subtrees of G and G* , we then have det A = A t(G) + 0(A 2 ), 
det A * = A t(G*) + 0( A 2 ). (The number of oriented subtrees of a balanced graph is the 
same for any given root, by exercise 22, but we do not need that fact.) 

If we group vertices Vj k for equal k the matrix A* can be partitioned as shown 
above. Let B k y be the submatrix of A* consisting of the rows for V jk and the columns 
for Vj/ fc /, for all j and j' such that V jk and V,/*,/ are in G*. By adding the 2nd, . . . , 
mth columns of each submatrix to the first column and then subtracting the first row 
of each submatrix from the 2nd, . . . , mth rows, the matrix A* is transformed so that 


/ dkk' 

0 

* 

0 . 

: () 

for k / k ' , 

to 

?r 

?r 

II 

/ A -\-a k k 

0 

* 

A+m 

. * \ 

0 

V o 

0 . 

. 02 



V 0 

0 

A+m / 


The asterisks in the top rows of the transformed submatrices turn out to be irrelevant, 
because the determinant of A * is now seen to be (A + m) (m-1)n times 


! A + a oo aoi 

flio A + an 


a 0(n-l) ^ 

^l(n — 1) 


\ a (n-l)0 ••• A + a( n _!)( n _!) / 


det 


Ai(G) + 0(A 2 ). 
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Notice that when n = 1 and there are m arcs from Vo to itself, we find in particular 
that exactly m rn ~ 1 oriented trees are possible on m labeled nodes. This result will be 
obtained by quite different methods in Section 2. 3. 4. 4. 

This derivation can be generalized to determine the number of oriented subtrees 
of G* when G is an arbitrary directed graph; see R. Dawson and I. J. Good, Ann. 
Math. Stat. 28 (1957), 946-956; D. E. Knuth, Journal of Combinatorial Theory 3 
(1967), 309-314. An alternative, purely combinatorial proof has been given by J. B. 
Orlin, Journal of Combinatorial Theory B25 (1978), 187-198. 

22. The total number is (a\ + ■ • • + <r n ) times the number of Eulerian trails starting 
with a given edge ei, where init(ei) = Vi. Each such trail determines an oriented 
subtree with root Vi by Lemma E, and for each of the T oriented subtrees there are 
n"=i(°j _ 1)! paths satisfying the three conditions of Theorem D, corresponding to 
the different order in which the arcs {e | init(e) = V), e ^ e[Vj], e / ei} are entered 
into P. (Exercise 14 provides a simple example.) 

23. Construct the directed graph Gk with m k ~ l vertices as in the hint, and denote 
by [xi, . . ,,Xk\ the arc mentioned there. For each function that has maximum period 
length, we can define a unique corresponding Eulerian trail, by letting f(x i, . . . , Xk) = 
Xk+i if arc [x\,...,x k ] is followed by [x 2 , . . ■ , Xfc+i]. (We regard Eulerian trails as 
being the same if one is just a cyclic permutation of the other.) Now G k = G k _ 1 in 

the sense of exercise 21, so G k has m m 2 times as many oriented subtrees as 

Gk- 1 ; by induction G k has m m -1 oriented subtrees, and m m ' ~ k with a given 
root. Therefore by exercise 22 the number of functions with maximum period, namely 
the number of Eulerian trails of G k starting with a given arc, is m~ k (ml)™ 1 * [For 
m = 2 this result is due to C. Flye Sainte-Marie, L’ Intermediate des Mathematiciens 
1 (1894), 107-110.] 

24. Define a new directed graph having Ej copies of e 3 , for 0 < j < m. This graph 
is balanced, hence it contains an Eulerian trail (e 0 , . . . ) by Theorem G. The desired 
oriented path comes by deleting the edge e 0 from this Eulerian trail. 

25. Assign an arbitrary order to all arcs in the sets Ij = {e | init(e) = V,} and 
Fj = {e | fin(e) = Vj}. For each arc e in Ij, let ATAG(e) = 0 and ALINK(e) = e’ if 
e follows e in the ordering of Ij ; also let ATAG(e) = 1 and ALINK(e) = e if e is last 
in Ij and e is first in Fj. Let ALINK(e) = A in the latter case if Fj is empty. Define 
BLINK and BTAG by the same rules, reversing the roles of init and fin. 

Examples (using alphabetic order in each set of arcs): 




ad 1 
b A 1 
c / 1 
d A 1 
e A 1 

/ A 1 

r a 1 


c 

A 


u 

<c 

H 

n 

0 

0 

1 

0 

1 

1 

1 



s 

o 

S3 

U 

M 


1 — 1 



H 

-J 

H 


■=« 

CQ 

m 

c 

0 

b 

1 

a 

1 

d 

0 

e 

0 

a 

1 

h 

0 

f 

0 

9 

0 

f 

1 

j 

0 

d 

1 

c 

1 

h 

0 

b 

1 

j 

0 

e 

1 

A 

1 



2. 3. 4. 2 


ANSWERS TO EXERCISES 585 


Note: If in the oriented tree representation we add another arc from II to itself, 
we get an interesting situation: Either we get the standard conventions 2.3.1-(8) with 
LLINK, LTAG, ELINK, RTAG interchanged in the list head, or (if the new arc is placed 
last in the ordering) we get the standard conventions except RTAG = 0 in the node 
associated with the root of the tree. 

This exercise is based on an idea communicated to the author by W. C. Lynch. 
Can tree traversal algorithms like Algorithm 2.3. IS be generalized to classes of digraphs 
that are not oriented trees, using such a representation? 

27. Let aij be the sum of p(e) over all arcs e from V) to V). We are to prove that 

t] = f° r all j ■ Since J2i a ji = 1; we must prove that J2i a n t j = 

But this is not difficult, because both sides of the equation represent the sum of all 
products p(ei) . ..p(e n ) taken over subgraphs {ei, . . . , e„} of G such that init(ei) = V) 
and such that there is a unique oriented cycle contained in {ei, . . . , e„}, where this cycle 
includes Vj. Removing any arc of the cycle yields an oriented tree; the left-hand side of 
the equation is obtained by factoring out the arcs that leave Vj, while the right-hand 
side corresponds to those that enter Vj. 

In a sense, this exercise is a combination of exercises 19 and 26. 

28. Every term in the expansion is oi pl . ..a mPm &i 91 ...b nqn , where 0 < p t < n for 
1 < i < m and 0 < qj < m for 1 < j < n , times some integer coefficient. Represent 
this product as a directed graph on the vertices (0, m, . . . , u m , Vi , . . . , u„}, with arcs 
from m to v Pi and from Vj to u qj , where uo = v 0 = 0. 

If the digraph contains a cycle, the integer coefficient is zero. For each cycle 
corresponds to a factor of the form 

a ioio "jo ii a iiji ■ ■ ■ a ik-l3k-l bjk-lio (*) 

where the indices ■ ■ . ,U-i) are distinct and so are the indices (jo, ji, ■ ■ . ,jk- 1 ). 

The sum of all terms containing (*) as a factor is (*) times the determinant obtained 
by setting a it j <- [j=ji] for 0 < j < n and b jti «- [i = i (i+1) mod k ] for 0 < i < m, 
for 0 < l < k, leaving the variables in the other m + n - 2k rows unchanged. This 
determinant is identically zero, because the sum of rows io, ii, ..., i k - i in the top 
section equals the sum of rows jo, ji, ■ . . , jk-i in the bottom section. 

On the other hand, if the directed graph contains no cycles, the integer coefficient 
is +1. This follows because each factor a lPi and bj qj must have come from the diagonal 
of the determinant: If any off-diagonal element a; 0 j 0 is chosen in row io of the top 
section, we must choose some off-diagonal bj 0 i 1 from row j 0 of the bottom section, 
hence we must choose some off-diagonal a iljl from row i x of the top section, etc., 
forcing a cycle. 

Thus the coefficient is +1 if and only if the corresponding digraph is an oriented 
tree with root 0. The number of such terms (hence the number of such oriented trees) 
is obtained by setting each a t] and bji to 1; for example, 
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Vo 

0 
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3/ 


= ^ el ( 2 3) 4 ' 3 ' 3 - 

In general we obtain det( n + 1 x ) • (n + l) m_1 • (m + l) n_1 . 
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Notes: J. J. Sylvester considered the special case m = n and a io = 020 = 
••• = fflmo = 0 in Quarterly J. of Pure and Applied Math. 1 (1857), 42-56, where 
he conjectured (correctly) that the total number of terms is then n”(n + l) n_1 . He 
also stated without proof that the ( n + l) n_1 nonzero terms present when ciij = 5ij 
correspond to all connected cycle-free graphs on {0, 1, . . . , n}. In that special case, he 
reduced the determinant to the form in the matrix tree theorem of exercise 19, e.g., 

( bio + 612 + 613 — 612 —6x3 

—621 620 + 621 + 623 —623 

—631 —632 630 + 631 + 632 

Cayley quoted this result in Crelle 52 (1856), 279, ascribing it to Sylvester; thus it is 
ironic that the theorem about the number of such graphs is often attributed to Cayley. 

By negating the first m rows of the given determinant, then negating the first m 
columns, we can reduce this exercise to the matrix tree theorem. 

[Matrices having the general form considered in this exercise are important in 
iterative methods for the solution of partial differential equations, and they are said 
to have “Property M.” See, for example, Louis A. Hageman and David M. Young, 
Applied Iterative Methods (Academic Press, 1981), Chapter 9.] 
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1. The root is the empty sequence; arcs go from (xi, . . . , x„) to (xi, . . . , x„-\). 


2. Take one tetrad type and rotate it 180° to get another tetrad type; these two types 
clearly tile the plane (without further rotations), by repeating a 2 x 2 pattern. 


3. Consider the set of tetrad types 


for all positive integers j . The right half 


plane can be tiled in uncountably many ways; but whatever square is placed in the 
center of the plane puts a finite limit on the distance it can be continued to the left. 


4. Systematically enumerate all possible ways to tile an nxn block, for n = 1,2,..., 
looking for toroidal solutions within these blocks. If there is no way to tile the plane, the 
infinity lemma tells us there is an n with no nxn solutions. If there is a way to tile the 
plane, the assumption tells us that there is an n with an nxn solution containing a rect- 
angle that yields a toroidal solution. Hence in either case the algorithm will terminate. 

[But the stated assumption is false, as shown in the next exercise; and in fact there 
is no algorithm that will determine in a finite number of steps whether or not there 
exists a way to tile the plane with a given set of types. On the other hand, if such a 
tiling does exist, there is always a tiling that is quasitoroidal, in the sense that each of 
its n x n blocks occurs at least once in every f(n ) x f(n) block, for some function /. 
See B. Durand, Theoretical Computer Science 221 (1999), 61-75.] 

5. Start by noticing that we need classes “ ^ replicated in 2 x 2 groups in any solution. 
Then, step 1: Considering just the a squares, show that the pattern “ b , must be 
replicated in 2 x 2 groups of a squares. Step n > 1: Determine a pattern that must 
appear in a cross-shaped region of height and width 2 n — 1. The middle of the crosses 
has the pattern ^ replicated throughout the plane. 

For example, after step 3 we will know the contents of 7 x 7 blocks throughout the 
plane, separated by unit length strips, every eight units. The 7x7 blocks that are of 
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class Na in the center have the form 


aa 

PKQ 

ab 

PQP 

aa 

PBK 

ab 

7 PJ 

5Na 

7 RB 

SQK 

7 LJ 

SNb 

yPB 

ac 

/3DS 

ad 

PQTY 

ac 

/ IBS 

ad 

yPQ 

5PJ 

jPXB 

SNa 

'yRQ 

SRB 

•yRB 

aa 

/3UK 

ab 

1 3DP 

aa 

f3BK 

ab 

- yTJ 

SNc 

'ySB 

SDS 

'ySJ 

5Nd 

jTB 

ac 

PQS 

ad 

fdDT 

ac 

f)BS 

ad 


The middle column and the middle row is the “cross” just filled in during step 3; the 
other four 3x3 squares were filled in after step 2; the squares just to the right and 
below this 7x7 square are part of a 15 x 15 cross to be filled in at step 4. 

For a similar construction that leads to a set of only 35 tetrad types having nothing 
but nontoroidal solutions, see R. M. Robinson, Inventiones Math. 12 (1971), 177-209. 
Robinson also exhibits a set of six squarish shapes that tile the plane only nontoroidally, 
even when rotations and reflections are allowed. In 1974, Roger Penrose discovered a 
set of only two polygons, based on the golden ratio instead of a square grid, that tile the 
plane only aperiodically; this led to a set of only 16 tetrad types with only nontoroidal 
solutions [see B. Griinbaum and G. C. Shephard, Tilings and Patterns (Freeman, 1987), 
Chapters 10-11; Martin Gardner, Penrose Tiles to Trapdoor Ciphers (Freeman, 1989), 
Chapters 1-2]. 

6. Let k and m be fixed. Consider an oriented tree whose vertices each represent, 
for some n, one of the partitions of {l,...,n} into k parts, containing no arithmetic 
progression of length m. A node that partitions {1, . . . ,n + 1} is a child of one for 
{1, . . . , n} if the two partitions agree on {1, . . . , n}. If there were an infinite path to 
the root we would have a way to divide all integers into k sets with no arithmetic 
progression of length m. Hence, by the infinity lemma and van der Waerden’s theorem, 
this tree is finite. (If k = 2, m = 3, the tree can be rapidly calculated by hand, and the 
least value of N is 9. See Studies in Pure Mathematics, ed. by L. Mirsky (Academic 
Press, 1971), 251-260, for van der Waerden’s interesting account of how the proof of 
his theorem was discovered.) 

7 . The positive integers can be partitioned into two sets S 0 and Si such that neither 
set contains any infinite computable sequence (see exercise 3.5-32). So in particular 
there is no infinite arithmetic progression. Theorem K does not apply because there is 
no way to put partial solutions into a tree with finite degrees at each vertex. 

8. Let a “counterexample sequence” be an infinite sequence of trees that violates 
Kruskal’s theorem, if such sequences exist. Assume that the theorem is false; then let 
Ti be a tree with the smallest possible number of nodes such that T\ can be the first 
tree in a counterexample sequence; if 7\ , . . . , Tj have been chosen, let T j+1 be a tree 
with the smallest possible number of nodes such that Ti, . . . ,Tj, T J+1 is the beginning 
of a counterexample sequence. This process defines a counterexample sequence (T n ). 
None of these T” s is just a root. Now, we look at this sequence very carefully: 

(a) Suppose there is a subsequence T ni , T„ 2 , ... for which l(T ni ), l(T n2 ), ... 
is a counterexample sequence. This is impossible; otherwise Ti, ..., T ni ~ i, l(T ni ), 
l(T n2 ), . . . would be a counterexample sequence, contradicting the definition of T ni . 
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(b) Because of (a), there are only finitely many j for which l(Tj) cannot be 
embedded in l(Tk ) for any k > j. Therefore by taking rii larger than any such j 
we can find a subsequence for which l{T ni ) C l(T„ 2 ) C l(T„ 3 ) C • • • . 

(c) Now by the result of exercise 2.3.2-22, r(T nj ) cannot be embedded in r(T„ k ) 
for any k > j, else T nj C T nk . Therefore Ti, ..., T ni -i, r{T ni ), r(T„ 2 ), ... is a 
counterexample sequence. But this contradicts the definition of T ni . 

Notes: Kruskal, in Trans. Amer. Math. Soc. 95 (1960), 210-225, actually proved 
a stronger result, using a weaker notion of embedding. His theorem does not follow 
directly from the infinity lemma, although the results are vaguely similar. Indeed, Konig 
himself proved a special case of Kruskal’s theorem, showing that there is no infinite 
sequence of pairwise incomparable n-tuples of nonnegative integers, where comparabil- 
ity means that all components of one n-tuple are < the corresponding components of 
the other [Matematikai es Fizikai Lapok 39 (1932), 27-29]. For further developments, 
see J. Combinatorial Theory A13 (1972), 297-305. See also N. Dershowitz, Inf. Proc. 
Letters 9 (1979), 212-215, for applications to termination of algorithms. 

SECTION 2. 3. 4. 4 

1. \nA(z) = lnz + ^a fc ln( r -^) = lnz + = ln* + >T^-l 

k>l k,t>l t>l 

2. By differentiation, and equating the coefficients of z n , we obtain the identity 

na n +i — ^ 

fc>l d\k 

Now interchange the order of summation. 

4. (a) A(z) certainly converges at least for \z\ < i, since a n is less than the number 
of ordered trees fe n -i- Since A(l) is infinite and all coefficients are positive, there is a 
positive number a < 1 such that A(z) converges for \z\ < a, and there is a singularity 
at z = a. Let ip(z) = A(z)/z; since ip(z) > e z * (z \ we see that ip(z) = m implies 
2 < In m/m, so ip(z) is bounded and lim--^- ip(z) exists. Thus a < 1, and by Abel’s 
limit theorem a — a - exp(a + | A(a 2 ) + | A(a 3 ) + •••). 

(b) A(z 2 ), A(z 3 ), ... are analytic for |z| < y/a, and |A(z 2 )-|-|A(z 3 )-| converges 

uniformly in a slightly smaller disk. 

(c) If dF/dw = a — 1/0, the implicit function theorem implies that there is an 
analytic function f(z ) in a neighborhood of (a, a/a) such that F(z, f(z)) = 0. But this 
implies f(z ) = A(z)/z, contradicting the fact that A(z) is singular at a. 

(d) Obvious. 

(e) dF/dw = A(z) — 1 and |A(z)| < A(a) = 1, since the coefficients of A(z) are 
all positive. Hence, as in (c), A(z) is regular at all such points. 

(f) Near (a, 1/a) we have the identity 0 = j3(z - a) + (a/2)(w - 1/a) 2 + higher 

order terms, where w = A(z)/z-, so w is an analytic function of y/z — a here by the 
implicit function theorem. Consequently there is a region \z\ < a\ minus a cut [a, ai] 
in which A{z) has the stated form. (The minus sign is chosen since a plus sign would 
make the coefficients ultimately negative.) . — 

(g) Any function of the stated form has coefficient asymptotically l2lL f 

Note that Q " \ n ) 

CfH (;(?))• 
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For further details, and asymptotic values of the number of free trees, see R. Otter, 
Ann. Math. (2) 49 (1948), 583-599. 


5. 


Therefore 


Cn - 

3 1+2J2 + " 


/ Cl + ji - 1 \ 

{ Cn jn 1 \ 

V ji ) " 

\ jn ) 


n > 1. 


2C(z) + l- z = (l- z) CI (1-^ 2 ) C2 {l-z 3 )~ C3 ... = exp(C(z)+~C{z 2 ) + ---). 

We find C(z) = z + z 2 + 2z 3 + 5z 4 + 12z 5 + 33z 6 + 90 z 7 + 2612® + 7662® + • • • . When 
n > 1, the number of series-parallel networks with n edges is 2c n [see P. A. MacMahon, 
Proc. London Math. Soc. 22 (1891), 330-339]. 

6. zG(z) 2 = 2 G(z) - 2 - zG(z 2 ); G(z) = 1 + z + z 2 + 2z 3 + 3z 4 + 6 2 5 + II 2 6 + 
23 2 7 + 462 s + 982® + • • • . The function F(z) = 1 — zG(z) satisfies the simpler relation 
F(z 2 ) = 2 2 + F(z) 2 . [J. H. M. Wedderburn, Annals of Math. 24 (1922), 121 140.] 

7. g n = ca n n _3/2 (l + 0(l/n)), where c « 0.7916031835775, a « 2.483253536173. 

8 . 



9. If there are two centroids, by considering a path from one to the other we find that 
there can’t be intermediate points, so any two centroids are adjacent. A tree cannot 
contain three mutually adjacent vertices, so there are at most two. 

10. If X and 1 are adjacent, let s(X, Y) be the number of vertices in the Y subtree 
of X. Then s(X, Y) + s(Y,X) = n. The argument in the text shows that if Y is a 
centroid, weight(X) = s(X, Y). Therefore if both X and Y are centroids, weight (X) = 
weight(X) = n/2. 

In terms of this notation, the argument in the text goes on to show that if 
s(X, V') > s(Y, X), there is a centroid in the Y subtree of X. So if two free trees 
with m vertices are joined by an edge between X and Y . we obtain a free tree in which 
s(X,Y) = m = s(Y, X), and there must be two centroids (namely X and Y). 

[It is a nice programming exercise to compute s(X,Y) for all adjacent X and Y 
in 0(n) steps; from this information we can quickly find the centroid(s). An efficient 
algorithm for centroid location was first given by A. J. Goldman, Transportation Sci. 
5 (1971), 212-221.] 

11. zT(zY = T(z) - 1; thus 2 + T(z)- 1 = T^) 1 "*. By Eq. 1.2.9-( 2 i), T(z) = 
^ ~2n ^n(l, ~t)z n , so the number of t-ary trees is 

f 1 + tn\ 1 _ / tn\ 1 

V n )l+tn \ n ) (t — l)n + 1 ’ 

12. Consider the directed graph that has one arc from V) to Vj for all i / j. The 
matrix A 0 of exercise 2.3.4.2-19 is a combinatorial (n — 1) x (n — 1) matrix with n — 1 
on the diagonal and —1 off the diagonal. So its determinant is 

{n + (n-l)(^l))n n ~ 2 = n n ~ 2 , 

the number of oriented trees with a given root. (Exercise 2.3.4.2-20 could also be used.) 
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13. 



14. True, since the root will not become a leaf until all other branches have been 
removed. 

15. In the canonical representation, Vi, V 2 , ... , V n _i, f(V n - i) is a topological sort of 
the oriented tree considered as a directed graph, but this order would not in general be 
output by Algorithm 2.2.3T. Algorithm 2.2.3T can be changed so that it determines 
the values of Vi, V 2 , . . . , V n -i if the “insert into queue” operation of step T6 is replaced 
by a procedure that adjusts links so that the entries of the list appear in ascending 
order from front to rear; then the queue becomes a priority queue. 

(However, a general priority queue isn’t needed to find the canonical represen- 
tation; we only need to sweep through the vertices from 1 to n, looking for leaves, 
while pruning off paths from new leaves less than the sweep pointer; see the following 
exercise.) 

16. Dl. Set C [1] C [n] <- 0, then set C[/(V;)] 4— C[/(V))]+1 for 1 < j < n. 

(Thus vertex k is a leaf if and only if C[fc] = 0.) Set k <- 0 and j 4 - 1. 

D2. Increase k one or more times until C[fc] = 0, then set l <— k. 

D3. Set PARENT [I] <- f(V 3 ), l <- f(V 3 ), C [/] <- C[Z] - 1, and j 4- j + 1. 

D4. If j = n, set PARENT [I] 4— 0 and terminate the algorithm. 

D5. If C[I] =0 and l < k, go to D3; otherwise go back to D2. | 

17. There must be exactly one cycle x i, x 2 , ■ . . , x k where f(xj) - x ]+1 and f(x k ) = x 2 . 
We will enumerate all / having a cycle of length k such that the iterates of each 
x ultimately come into this cycle. Define the canonical representation /(Vi), f(V 2 ), 
■ ■■, }(V m -k ) as in the text; now JiVm-k) is in the cycle, so we continue to get a 
“canonical representation” by writing down the rest 
of the cycle f{f(V m - k )), f(f(f{V m - k ))), etc. For 
example, the function with m = 13 whose graph is 
shown here leads to the representation 3, 1, 8, 8, 

1, 12, 12, 2, 3, 4, 5, 1. We obtain a sequence of 
m — 1 numbers in which the last k are distinct. Con- 
versely, from any such sequence we can reverse the 
construction (assuming that k is known); hence there 
are precisely m-m m ~ k ~ l such functions having a k- 
cycle. (For related results, see exercise 3.1-14. The 
formula m rn ~ 1 Q(m) was first obtained by L. Katz, 

Annals of Math. Statistics 26 (1955), 512-517.) 
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18. To reconstruct the tree from a sequence si, S2, ..., s n -i, begin with si as the 
root and successively attach arcs to the tree that point to si, s 2 , . . . ; if vertex s k has 
appeared earlier, leave the initial vertex of the arc leading to s k : nameless, otherwise 
give this vertex the name s k . After all n — 1 arcs 
have been placed, give names to all vertices that re- 
main nameless by using the numbers that have not 
yet appeared, assigning names in increasing order 
to nameless vertices in the order of their creation. 

For example, from 3, 1, 4, 1, 5, 9, 2, 6, 5 we 
would construct the tree shown on the right. There 
is no simple connection between this method and 
the one in the text. Several more representations 
are possible; see the article by E. H. Neville, Proc. 

Cambridge Phil. Soc. 49 (1953), 381-385. 

19. The canonical representation will have precisely n - k different values, so we 
enumerate the sequences of n — 1 numbers with this property. The answer is n "~ k . 

20 . Consider the canonical representation of such trees. We are asking how many 
terms of (x\ + • • ■ + x n ) n ~ 1 have k 0 exponents zero, ki exponents one, etc. This is 
plainly the coefficient of such a term times the number of such terms, namely 



(n - 1)! ^ n! 

(0!)*o(i!)fci . . . („!)*„ x k 0 \ k!\ . . . k n \ ' 


21. There are none with 2m vertices; if there are n = 2m + 1 vertices, the answer is 
obtained from exercise 20 with k 0 = m+ 1, k 2 = m, namely ( 2 ”^ 1 )(2rn)!/2 m . 

22. Exactly n n-2 ; for if X is a particular vertex, the free trees are in one-to-one 
correspondence with oriented trees having root X. 

23. It is possible to put the labels on every unlabeled, ordered tree in n! ways, and 
each of these labeled, ordered trees is distinct. So the total number is n\b n - 1 = 
(2n — 2)!/(n — 1)!. 


24. There are as many with one given root as with another, so the answer in general 
is 1/n times the answer in exercise 23; and in this particular case the answer is 30. 

25. For 0 < q < n, r(n,q) = (n — q)n q ~ l . (The special case s = 1 in Eq. (24).) 

26. (k = 7) 



5 


Blue 
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27. Given a function g from {1, 2, . . . , r} to {1, 2, . . . , q} such that adding arcs from V k 
to Ug (k) introduces no oriented cycles, construct a sequence ai, . . . , a r as follows: Call 
vertex V k “free” if there is no oriented path from Vj to V k for any j / k. Since there 
are no oriented cycles, there must be at least one free vertex. Let bi be the smallest 
integer for which V bl is free; and assuming that bi, . . . , b t have been chosen, let b t+ i 
be the smallest integer different from for, ..., b t for which Vi is free in the graph 
obtained by deleting the arcs from V bk to U g(bk) for 1 < k < t. This rule defines a 
permutation b k b 2 . . . b r of the integers {1,2,..., r-}. Let a k = g(b k ) for 1 < k < r; this 
defines a sequence such that 1 < a k < q for 1 < k < r, and 1 < a r < p. 

Conversely if such a sequence or, . . a r is given, call a vertex V k “free” if there 
is no j for which aj > p and f(aj) = k. Since a r < p there are at most r - 1 non- 
free vertices. Let b i be the smallest integer for which V bl is free; and assuming that 
bi, . . . , b t have been chosen, let 6 (+ i be the smallest integer different from bi, . . . , bt 
for which V bt+1 is free with respect to the sequence o t +i, . . . , a r . This rule defines a 
permutation M 2 • • b r of the integers {1,2 ,..., t-}. Let g(b k ) = a k for 1 < k < r; this 
defines a function such that adding arcs from V k to U g(k) introduces no oriented cycles. 

28. Let / be any of the n m_1 functions from {2, . . . , m) to {1, 2, . . . , n}, and consider 
the directed graph with vertices Ui, . . . , U m , Vi, . . . , V„ and arcs from U k to V f(k) for 
1 < k <m. Apply exercise 27 with p = 1, q = m, r = n, to show that there are m n ~ 1 
ways to add further arcs from the V’s to the t/’s to obtain an oriented tree with root 
Ui. Since there is a one-to-one correspondence between the desired set of free trees and 
the set of oriented trees with root Ui, the answer is n m ~ 1 m n ~ 1 . [This construction can 
be extensively generalized; see D. E. Knuth, Canadian J. Math. 20 (1968), 1077-1086.] 

29. If y = x l , then (tz)y = In y, and we see that it is sufficient to prove the identity 
for t = 1. Now if zx = lna: we know by exercise 25 that x m = E k (m, 1 )z k for 
nonnegative integers m. Hence 


r zxr 

x = e 


<r-v ( zxr) k 
2-* k\ 


^ r r k z k +iE j {k, 1 ) ^z k ^(k\ 

2 . fcj - 2 . fcf 3,1)1 

j,k k j 


[Exercise 4.7-22 derives considerably more general results.] 

30. Each graph described defines a set C x C {1,. . . ,n), where j is in C x if and only 

if there is a path from tj to r, for some i < x. For a given C x each graph described 
is composed of two independent parts: one of the x(x + t\Z\ + • • • + e n z n Y 1 + "' +en ~ l 
graphs on the vertices n, s jk , tj for i < x and j G C x , where ej = [j eC x ], plus one of 
the y (y + (1 — £ 1)21 + • • • + (1 — e„) 2: n )^ 1 ei * + 1 graphs on the other vertices. 

31. G(z) = z + G(z ) 2 + G(z) 3 + G(z) 4 + ■ ■ ■ = z + G(z) 2 /( 1 - G(z)). Hence G(z) = 
i(l + 2 — %/ 1 — 62 + z 2 ) = z + z 2 + 3 2 3 + II 2 4 + 452 s + • • • . [Notes: Another 
problem equivalent to this one was posed and solved by E. Schroder, Zeitschrift fur 
Mathematik und Physik 15 (1870), 361-376, who determined the number of ways to 
insert nonoverlapping diagonals in a convex (n + l)-gon. These numbers for n > 1 
are just half the values obtained in exercise 2.2.1-11, since Pratt’s grammar allows 
the root node of the associated parse tree to have degree one. The asymptotic value 
is calculated in exercise 2.2.1-12. Curiously, the value [ 2 10 ] G(z) = 103049 seems 
to have been calculated already by Hipparchus in the second century B.C., as the 
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number of “affirmative compound propositions that can be made from only ten simple 
propositions”; see R. P. Stanley, AMM 104 (1997), 344-350; F. Acerbi, Archive for 
History of Exact Sciences 57 (2003), 465-502.] 

32. Zero if no / 1 + n 2 + 2n 3 + 3n 4 + • • • (see exercise 2.3-21), otherwise 
(n 0 + ni + • • • + n m — l)!/no! n 4 ! . . . n m !. 

To prove this result we recall that an unlabeled tree with n = n 0 + m -\ \-n m 

nodes is characterized by the sequence d\ d 2 . . . d n of the degrees of the nodes in 
postorder (Section 2.3.3). Furthermore such a sequence of degrees corresponds to a 
tree if and only if 53* =1 (1 — dj) > 0 for 0 < k < n. (This important property of Polish 
postfix notation is readily proved by induction; see Algorithm 2.3.3F with / a function 
that creates a tree, like the TREE function of Section 2.3.2.) In particular, d\ must 
be 0. The answer to our problem is therefore the number of sequences d 2 . . . d n with 
rij occurrences of j for j > 0, namely the multinomial coefficient 

( n ~ 1 V 

\ ^0 1 5 Tl\ , . . . , Tim / 


minus the number of such sequences d 2 ...d n for which £*(1 ~ d j) < 0 for some 

k > 2. 3 

We may enumerate the latter sequences as follows: Let t be minimal such that 
X^j =2 (l — dj) < 0j then X]j= 2 (l — dj) = — s where 1 < s < dt, and we may form the 
subsequence d ' 2 . . . d' n = d t ~ 1 . . . d 2 0d (+ i . . . d n , which has rij occurrences of j for j ^ d t , 
rij - 1 occurrences of j for j = d t . Now Y!] = 2 (1 ~ d'-) is equal to d t when k = n, and 
equal to d t - s when k = t; when k < t, it is 

Y^(l-dj)- (l-dj)< ^ (l-dj)=d t -s-l. 

2 <j<t 2 <j<t-k 2 <j<t 

It follows that, given s and any sequence d' 2 ... d' n , the construction can be reversed; 
hence the number of sequences d 2 . . . d n that have a given value of d t and s is the 
multinomial coefficient 

( ). 

V Ho , ... , 7ld t 1 , . . . , Tim J 

The number of sequences d 2 . . . d n that correspond to trees is therefore obtained by 
summing over the possible values of d t and s: 




3 = 0 



( n ~ 1 )! 
n 0 !ni! . . .n m \ 


m 


Ed 


~j)n J 


and the latter sum is 1. 

An even simpler proof of this result has been given by G. N. Raney (Transactions of 
the American Math. Society 94 (1960), 441—451). If d\ d 2 . . . d n is any sequence with 
n i appearances of j, there is precisely one cyclic rearrangement dk . . . d„di . . . dk-i 
that corresponds to a tree, namely the rearrangement where k is maximal such that 
]Cj=i (1 — dj) is minimal. [This argument in the case of binary trees was apparently 
first discovered by C. S. Peirce in an unpublished manuscript; see his New Elements of 
Mathematics 4 (The Hague: Mouton, 1976), 303-304. It was discovered in the case of 
t-ary trees by Dvoretzky and Motzkin, Duke Math. J. 14 (1947), 305-313.] 

Still another proof, by G. Bergman, inductively replaces dkdk+i by (dk + dk+i — 1) 
if dk > 0 [ Algebra Universalis 8 (1978), 129-130], 
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The methods above can be generalized to show that the number of (ordered, 
unlabeled) forests having / trees and rij nodes of degree j is (n - 1)! //no! raj! . . . n m !, 
provided that the condition n 0 = f + n 2 + 2 n 3 + • ■ ■ is satisfied. 

33. Consider the number of trees with n i nodes labeled 1, n 2 nodes labeled 2, , and 

such that each node labeled j has degree ej. Let this number be c(ni, n 2 , . . . ), with the 
specified degrees e \ , e 2 , ... regarded as fixed. The generating function G(zi , z 2 , . . . ) = 
]Cc(ni,ra 2 , • • • )zi 1 z% 2 . . . satisfies the identity G = ziG ei + ■ ■ ■ + z r G Cr , since ZjG e i 
enumerates the trees whose root is labeled j. And by the result of the previous exercise, 


c(ni,n 2 , . 


(m + n 2 H 1)! 

ni! n 2 ! . . . 

0 , 


if (1 — si)ni + (1 — e 2 )n 2 + • • • = 1; 
otherwise. 


More generally, since G f enumerates the number of ordered forests having such 
labels, we have for integer / > 0 


w 


f 


E 

/=(!— ei)ni + (l — e2)n 2 + -" 


(ni + n 2 H 1)! / 

ni! n 2 ! . . . 




n 2 

2 • • • • 


These formulas are meaningful when r = oo, and they are essentially equivalent to 
Lagrange’s inversion formula. 


SECTION 2. 3. 4. 5 

1. There are (®) in all, since the nodes numbered 8, 9, 10, 11, 12 may be attached in 
any of eight positions below 4, 5, 6, and 7. 



3. By induction on m, the condition is necessary. Conversely if )T " l = x 2~ lj = 1, we 
want to construct an extended binary tree with path lengths h, ..., l rn . When m = 1, 
we have h = 0 and the construction is trivial. Otherwise we may assume that the Vs 
are ordered so that l\ = l 2 = • • • = l q > > / 9+2 > • • • > l m > 0 for some q with 

1 < q < m. Now 2 ll_1 = i 1 = \q + integer, hence q is even. By induction 

on m there is a tree with path lengths h - 1, h, U, ■■■, l m ; take such a tree and 
replace one of the external nodes at level l\ — 1 by an internal node whose children are 
at level l\ = l 2 . 
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4. First, find a tree by Huffman’s method. If w 3 < w 3+1 , then l 3 > l j+1 , since the 
tree is optimal. The construction in the answer to exercise 3 now gives us another 
tree with these same path lengths and with the weights in the proper sequence. For 
example, the tree (n) becomes 



k+l=n — 1 
r+s+n— l=p 

(b) Take the partial derivative with respect to w: 


2 zB(w,wz)(B w (w,wz) + zB z (w, wz)) = B w (w,z). 

Therefore if H(z) = B w (l,z) = J2 n h n z n , we 6nd H(z) = 2 zB(z)(H{z) + zB' {z)\, and 
the known formula for B(z) implies 



so 


h n = 4" 


3n + 1 /2 n\ 
n + 1 V n ) 


The average value is h n /b n . (c) Asymptotically, this comes to n^fivn - 3n + 0(y/n). 

For the solution to similar problems, see John Riordan, IBM J. Res. and Devel. 4 
(1960), 473-478; A. Renyi and G. Szekeres, J. Australian Math. Soc. 7 (1967), 497-507; 
John Riordan and N. J. A. Sloane, J. Australian Math. Soc. 10 (1969), 278-282; and 
exercise 2.3.1-11. 


6. n + s — 1 = tn. 


7. E = (t — 1)1 + tn. 


8. Summation by parts gives X^fc=iU°St((^ ~ 1)&)J = nq — k, where the sum on 

the right is over values of k such that 0 < k < n and (t - l)k + 1 = t J for some j. The 
latter sum may be rewritten ~ l)/(£ — 1). 

9. Induction on the size of the tree. 


10. By adding extra zero weights, if neces- 
sary, we may assume that m mod (t — 1) = 1. 
To obtain a t-ary tree with minimum weighted 
path length, combine the smallest t values at 
each step and replace them by their sum. The 
proof is essentially the same as the binary case. 
The desired ternary tree is shown. 

F. K. Hwang has observed [SIAM J. Appl. 
Math. 37 (1979), 124-127] that a similar pro- 
cedure is valid for minimum weighted path 
length trees having any prescribed multiset of 
degrees: Combine the smallest t weights at 
each step, where t is as small as possible. 
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11. The “Dewey” notation is the binary representation of the node number. 

12. By exercise 9, it is the internal path length divided by n, plus 1. (This result holds 
for general trees as well as binary trees.) 

13. [See J. van Leeuwen, Proc. 3rd International Colloq. Automata, Languages and 
Programming (Edinburgh University Press, 1976), 382-410.] 

Hi. [Initialize.] Set A[m - 1 + i] <- Wi for 1 < i < m. Then set A[2m] oo, 
x <- m, i «— m + 1, j «— m — 1, k m. (During this algorithm A[i\ < ■ ■ • < 
A[2m - 1] is the queue of unused external weights; A[k\ > ■ ■ ■ > A[j ] is the 
queue of unused internal weights, empty if j < k; the current left and right 
pointers are x and y.) 

H2. [Find right pointer.] If j < k or A[i] < A[j], set y <- i and i <r- i+ 1; otherwise 
set y 4— j and j <— j — 1. 

H3. [Create internal node.] Set k <- k- 1, L[k] 4- x, R[k ] <- y, A[fc] e- A[x]+A[y], 
H4. [Done?] Terminate the algorithm if k = 1. 

H5. [Find left pointer.] (At this point j > k and the queues contain a total of k 
unused weights. If A[y] < 0 we have j = k, i = y + 1, and A[i\ > A[j].) If 
A[i\ < A[j], set x i and t «— i + 1; otherwise set x «— j and j j — 1. 
Return to step H2. | 

14. The proof for k = rn - 1 applies with little change. [See SIAM J. Appl. Math. 21 
(1971), 518.] 

15. Use the combined-weight functions (a) 1 + max(u)i,m 2 ) and (b) xwi + xw 2 , re- 

spectively, instead of tin + w 2 in (9). [Part (a) is due to M. C. Golumbic, IEEE Trans. 
C-25 (1976), 1164-1167; part (b) to T. C. Hu, D. Kleitman, and J. K. Tamaki, SIAM 
J. Appl. Math. 37 (1979), 246-256. Huffman’s problem is the limiting case of (b) as 
x -A 1, since £](1 + e) l ^Wj =J2 w j + 0{e 2 ).} 

D. Stott Parker, Jr., has pointed out that a Huffman-like algorithm will also find 

the minimum of w x x h + b w m x lrn when 0 < x < 1, if the two maximum weights 

are combined at each step as in part (b). In particular, the minimum of w\2~ h -) 1- 

w m 2~ lm , when mi < • • • < w m , is wi/2 + • • • + w m -i/2 rn ~ 1 + w rn /2 m ~ 1 . See D. E. 
Knuth, J. Comb. Theory A32 (1982), 216-224, for further generalizations. 

16. Let l m+1 = l' m+1 = 0. Then 

ra m m km km 

y. w ih ^ y w jij = y{ik - 4+0 y ^ < yxik - 4+1) y ^ = y , 

J = 1 1=1 k= 1 j = l k = 1 j = l j = l 

since lj > l' + 1 as in exercise 4. The same proof holds for many other kinds of optimum 
trees, including those of exercise 10. 

17. (a) This is exercise 14. (b) We can extend f(n) to a concave function f(x), so the 

stated inequality holds. Now F(m ) is the minimum of where the s 3 are 

internal node weights of an extended binary tree on the weights 1, 1, . . . , 1. Huffman’s 
algorithm, which constructs the complete binary tree with m - 1 internal nodes in this 
case, yields the optimum tree. The choice k = 2 rig(n / 3 ^ defines a binary tree with the 
same internal weights, so it yields the minimum in the recurrence, for each n. [SIAM J. 
Appl. Math. 31 (1976), 368-378.] We can evaluate F(n) in O(logn) steps; see exercises 
5.2.3-20 and 5.2.3-21. If f(n) is convex instead of concave, so that A 2 /(n) > 0, the 
solution to the recurrence is obtained when k = [n/2]. 
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SECTION 2. 3. 4. 6 

1. Choose one edge of the polygon and call it the base. Given a triangulation, let the 
triangle on the base correspond to the root of a binary tree, and let the other two sides 
of that triangle define bases of left and right subpolygons, which correspond to left 
and right subtrees in the same way. We proceed recursively until reaching “2-sided” 
polygons, which correspond to empty binary trees. 

Stating this correspondence another way, we can label the non-base edges of a 
triangulated polygon with the integers 0, . . . , n; and when two adjacent sides of a 
triangle are labeled a and /3 in clockwise order, we can label the third side ( a/3 ). The 
label of the base then characterizes the binary tree and the triangulation. For example, 



corresponds to the binary tree shown in 2.3.1-(i). [See H. G. Forder, Mathematical 
Gazette 45 (1961), 199-201.] 

2. (a) Take a base edge as in exercise 1, and give it d descendants if that edge is part 
of a (d + l)-gon in the dissected r-gon. The other d edges are then bases for subtrees. 
This defines a correspondence between Kirkman’s problem and all ordered trees with 
r — 1 leaves and k + 1 nonleaves, having no nodes of degree 1. (When k = r — 3 we 
have the situation of exercise 1.) 

(b) There are (£+([) ( r “ 3 ) sequences d\d 2 . . . d r+k of nonnegative integers such that 
r — 1 of the d's are 0, none of them are 1, and the sum is r + k — 1. Exactly one of 
the cyclic permutations d\d 2 . . . d r +k, d 2 ■ ■ ■ d r+k d\, . . . , d r +kd\ . . . d r+k - 1 satisfies the 
additional property that 5^? =1 (1 — dj) > 0 for 1 < q < r + k. 

[Kirkman gave evidence for his conjecture in Philos. Trans. 147 (1857), 217-272, 
§22. Cayley proved it in Proc. London Math. Soc. 22 (1891), 237-262, without noticing 
the connection to trees.] 

3. (a) Let the vertices be {1,2, ...,n}. Draw an RLINK from i to j if i and j are 
consecutive elements of the same part and i < j; draw an LLINK from j to j + 1 if 
j + 1 is the smallest of its part. Then there are k - 1 nonnull LLINKs, n — k nonnull 
RLINKs, and we have a binary tree whose nodes are 12... n in preorder. Using the 
natural correspondence of Section 2.3.2, this rule defines a one-to-one correspondence 
between “partitions of an n-gon’s vertices into k noncrossing parts” and “forests with 
n vertices and n — k + 1 leaves.” Interchanging LLINK with RLINK also gives “forests 
with n vertices and k leaves.” 

(b) A forest with n vertices and k leaves also corresponds to a sequence of nested 
parentheses, containing n left parentheses, n right parentheses, and k occurrences of 
“()”. We can enumerate such sequences as follows: 
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Say that a string of Os and Is is an (m, n, k) string if there are m Os, n Is, and 
k occurrences of “01”. Then 0010101001110 is a (7, 6, 4) string. The number of (m, n, k) 
strings is (™) (£), because we are free to choose which 0s and Is will form the 01 pairs. 

Let S(a) be the number of 0s in a minus the number of Is. We say that a string a 
is good if S(a) > 0 whenever a is a prefix of a (in other words, if a = a/3 implies that 
S(a) > 0); otherwise a is bad. The following alternative to the “reflection principle” of 
exercise 2.2. 1-4 establishes a one-to-one correspondence between bad ( n,n,k ) strings 
and arbitrary (n - 1, n + 1, k) strings: 

Any bad (n, n, k ) string a can be written uniquely in the form a = a0/3, where 
Q and 0 are good. (Here a R is the string obtained from a by reversing it and 
complementing all the bits.) Then a' = al/3 is an (n - 1, n + 1, k) string. Conversely, 
every (n - l,n + l,fc) string can be written uniquely in the form al/3 where a R and 0 
are good, and aO/3 is then a bad (n, n, k) string. 

Thus the number of forests with n vertices and k leaves is (") (”) - ("- 1 ) ("+ 1 ) = 
(fc — i ) (fc) ( k ) (fc-i) = n! (n — \)\/(n — k + 1)! ( n — k)\k\ (k — 1)!, a so-called Narayana 
number [T. V. Narayana, Comptes Rendus Acad. Sci. 240 (Paris, 1955), 1188-1189]. 

Notes: G. Kreweras, Discrete Math. 1 (1972), 333-350, enumerated noncrossing 
partitions in a different way. The partial ordering of partitions by refinement leads to an 
interesting partial ordering of forests, different from the one discussed in exercise 2.3.3- 
19; see Y. Poupard, Cahiers du Bureau Univ. de Recherche Oper. 16 (1971), Chapter 8; 
Discrete Math. 2 (1972), 279-288; P. Edelman, Discrete Math. 31 (1980), 171-180, 40 
(1982), 171-179; N. Dershowitz and S. Zaks, Discrete Math. 64 (1986), 215-218. 

A third way to define a natural lattice ordering of forests was introduced by 
R. Stanley in Fibonacci Quarterly 13 (1975), 215-232: Suppose we represent a forest 
by a string o of 0s and Is representing left and right parentheses as above; then a < a' 
if and only if S(a k ) < S(cr' k ) for all k, where o k denotes the first k bits of cr. Stanley’s 
lattice is distributive, unlike the other two. 

4. Let m = n A 2; by exercise 1, we want a correspondence between triangulated 
m-gons and (m - l)-rowed friezes. First let’s look more closely at the previous cor- 
respondence, by giving a “top-down” labeling to the edges of a triangulation instead 
of the “bottom-up” one considered earlier: Assign the empty label e to the base, then 
recursively give the labels cxL and aR to the opposite edges of a triangle whose base is 
labeled a. For example, the previous diagram becomes 



under these new conventions. If the base edge in this example is called 10, while the 
other edges are 0 to 9 as before, we can write 0 = 10 LLL, 1 = 10 LLR, 2 = 10 LR, 


2. 3. 4. 6 


ANSWERS TO EXERCISES 599 


3 = 10 RLL, etc. Any of the other edges can also be chosen as the base; thus, if 0 is 
chosen we have 1 = 0L, 2 = 0 RL, 3 = ORRLLL, etc. It is not difficult to verify that 
if u = va we have v = ua T , where a T is obtained by reading a from right to left and 
interchanging L with R. For example, 10 = ORRR = 1 LRR = 2 LR = 3RRL, etc. If 
u, v, and w are edges of the polygon with w = uaLy and w = v/3R-y, then u = v/3La T 
and v = uaR/3 T . 

Given a triangulation of a polygon whose edges are numbered 0, 1, . . . , m — 1, we 
define ( u , v) for any pair of distinct edges u and v as follows: Let u = va, and interpret 
a as a 2 x 2 matrix by letting L = (J J) and R = (J °). Then (u,v) is defined to 
be the element in the upper left corner of a. Notice that a T is the transpose of the 
matrix a, since R = L T ; hence we have (v,u) — (u,v). Notice also that ( u,v ) = 1 if 
and only if u _ and V- are joined by an edge of the triangulation, where U- denotes 
the vertex between edges u and u — 1. 

Let («, u) = 0 for all polygon edges u. We can now prove that v = ua implies 


f (u,v) (u,v + 1) \ 

V (u + 1, v) (u + l,v + 1) / 


for all u / v, 


(*) 


where u + 1 and v + 1 are the clockwise successors of u and v. The proof is by induction 
on m: Eq. (*) is trivial when m = 2, since the two parallel edges u and v are then 
related by u = ve, and a = e is the identity matrix. If any triangulation is augmented 
by extending some edge v with a triangle vv' v" , then v = ua implies v' = uaL and 
v" = uaR\ hence (u,v') and ( u,v ") in the extended polygon are respectively equal to 
(u,v) and (u,v) + (u, v + 1) in the original one. It follows that 


aL = 


( M 

\{u+ !,«') 


(«,«") \ 
(« + l ,v")J 


and 


( («,«" 

V(u+ l,v") 


(u,v" + l) \ 
(u + 1, v" + 1 ) J ’ 


and (*) remains true in the extended polygon. 

The frieze pattern corresponding to the given triangulation is now defined to be 
the periodic sequence 


(0,1) (1,2) (2,3) 

(0,2) (1,3) (2,4) 

(m— 1,2) (0,3) (1,4) 

(m-1,3) (0,4) (1,5) 


(m— 1,0) (0,1) (1,2) 

. (m — 1,1) (0,2) (1,3) 

(m-2,1) (m— 1,2) (0,3) 

(m- 2,2) (m-1,3) (0,4) 


and so on until m — 1 rows have been defined; the final row begins with ([m/2] + 1, 
[m/2]) when m > 3. Condition (*) proves that this pattern is a frieze, namely that 

(u, v)(u + l,v + 1) — (u,v + l)(u + 1, v) = 1, (**) 

because det L = det R = 1 implies deta = 1. Our example triangulation yields 

111111111111111111111 
124215131431242151314 ... 

2177144223 11 2177144223 ... 

13 12 3337158713 12 3337158 ... 

325582532 13 5325582532 13 ... 

532 13 5325582532 13 532558 
37158713 12 3337158713 12 3 
4223 11 2177144223 11 21771 
513143124215131431242 ... 

111111111111111111111 ... 
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The relation ( u , v) = 1 defines the edges of the triangulation, hence different triangu- 
lations yield different friezes. To complete the proof of one-to-one correspondence, we 
must show that every ( m — l)-rowed frieze pattern of positive integers is obtained in 
this way from some triangulation. 

Given any frieze of m - 1 rows, extend it by putting a new row 0 at the top and 
a new row m at the bottom, both consisting entirely of zeros. Now let the elements of 
row 0 be called (0, 0), (1,1), (2, 2), etc., and for all nonnegative integers u < v < u + m 
let (u, v) be the element in the diagonal southeast of (u, u) and in the diagonal southwest 
of (v,v). By assumption, condition (**) holds for all u < v < u + m. We can in fact 
extend (**) to the considerably more general relation 


(t,u)(v,w) + (t,w)(u,v) = (t,v)(u,w) for t < u < v < w < t + m. (***) 

For if (***) is false, let ( t,u,v,w ) be a counterexample with the smallest value of 
( w — t)m + u — t + w — v. Clearly t ^ u and r / m. Case 1: t + 1 < u. Then (***) 
holds for ( t , t + 1, v, w), (£, t + l,it, u), and ( t + 1, u, v, w), so we find ((t, u)(v, w) + 
( t , w )( u > v )){t + 1, v) = (t, v)(u, w)(t + 1, v ); this implies ( t + 1, v) = 0, a contradiction. 
Case 2: u+1 < w. Then (***) holds for (£, u,w-l,w), ( u , v, w—l,w), and (t, u, v, w— 1); 
we obtain a similar contradiction ( u , w — 1) = 0. Case 3: u = t+ 1 and w = v + 1. In 
this case (***) reduces to (**). 

Now we set u = t + 1 and w = t + m in (***), obtaining (t,v) = (v,t + m) for 
t < v < t + m, because (£ + 1, t + m) = 1 and (f, t + m) = 0. We conclude that the 
entries of any ( m. - l)-rowed frieze are periodic: ( u , v) = (v,u + m) = (u + m, v + m) = 
(v + m,u + 2m) = • • • . 

Every frieze pattern of positive integers contains a 1 in row 2. For if we set t = 0. 
v = u + 1, and w = u + 2 in (***) we get (0, u + l)(u, u + 2) = (0, u) + (0, u + 2), 
hence (0,u + 2) — (0, u + 1) > (0, u + 1) — (0,u) if and only if (u,u + 2) > 2. This 
cannot hold for all u in the range 0 < u < m - 2, because (0, 1) - (0,0) = 1 and 
(0, m) — (0, m — 1) = —1. 

Finally, if m > 3 we cannot have two consecutive Is in row 2, because (u, u + 2) = 
(u + 1, « + 3) = 1 implies ( u , u + 3) = 0. Therefore we can reduce the frieze to another 
one with m reduced by 1, as illustrated here for 7 rows reduced to 6: 


11111111... 
a b c d + 1 1 e+1 y z ... 
p q c+r d e u+y v w ... 
u q+ v r s u q+v r s 
u+y v w p q c+r d e ... 
y z a b c d + 11 e+1 . . . 
11111111. 


1111111 ... 

a b c d e y z 
p q r s u v w . . . 
u v w p q r s 
y z a b c d e . 

1111111 


The reduced frieze corresponds to a triangulation, by induction, and the unreduced 
frieze corresponds to attaching one more triangle. [Math. Gazette 57 (1974), 87-94, 
175-183; Conway and Guy, The Book of Numbers (New York: Copernicus. 1996). 
74-76, 96-97, 101-102.] 

Notes: This proof demonstrates that the function (u, v), which we defined on any 
triangulation via 2x2 matrices, satisfies (***) whenever ( l , u . v . 'u: ) are edges of the 
polygon in clockwise order. We can express each ( u , v ) as a polynomial in the numbers 
a 3 = ti ~ 1 ,j + 1); these polynomials are essentially identical to the “continuants” 
discussed in Section 4.5.3, except for the signs of individual terms. In fact, (j, k) = 
ji k +i iaj +2 , . . . , iak-i). Thus (***) is equivalent to Euler’s identity 
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for continuants in the answer to exercise 4.5.3-32. The matrices L and R have the 
interesting property that any 2x2 matrix of nonnegative integers with determinant 1 
can be expressed uniquely as a product of L’s and R’s. 

Many other interesting relationships are present; for example, the numbers in 
row 2 of an integer frieze count the number of triangles touching each vertex of the 
corresponding triangulated polygon. The total number of occurrences of (u, v) = 1 in 
the basic region 0<u<v-l<m-l and (u,v) / (0, m — 1) is the number of 
diagonals (chords) of the triangulation, namely m — 3 = n — 1. The total number of 2s 
is also n — 1, because (u,v) = 2 if and only if u~ and V- are opposing vertices of the 
two triangles adjacent to a chord. 

Another interpretation of (u,v) was found by D. Broline, D. W. Crowe, and I. M. 
Isaacs [Geometriae Dcdicata 3 (1974), 171-176]: It is the number of ways to match the 
v — u — 1 vertices between edges u and v — 1 with distinct triangles adjacent to those 
vertices. 

SECTION 2.3.5 

1. A List structure is a directed graph in which the arcs leaving each vertex are 
ordered, and where some of the vertices that have out-degree 0 are designated “atoms.” 
Furthermore there is a vertex 5 such that there is an oriented path from S to V for all 
vertices V / S. (With directions of arcs reversed, S would be a “root.”) 

2. Not in the same way, since thread links in the usual representation lead back to 
"PARENT.” which is not unique for sub-Lists. The representation discussed in exercise 
2.3.4.2-25, or some similar method, could perhaps be used (but this idea has not yet 
been exploited at the time of writing). 

3. As mentioned in the text, we prove also that P = PO upon termination. If only 
PO is to be marked, the algorithm certainly operates correctly. If n > 1 nodes are 
to be marked, we must have ATOM(PO) = 0. Step E4 then sets ALINK(PO) -f- A and 
executes the algorithm with PO replaced by ALINK(PO) and T replaced by PO. By 
induction (note that since MARK(PO) is now 1, all links to PO are equivalent to A by 
steps E4 and E5), we see that ultimately we will mark all nodes on paths that start with 
ALINK(PO) and do not pass through PO; and we will then get to step E6 with T = PO and 
P = ALINK(PO) . Now since ATOM(T) = 1, step E6 restores ALINK(PO) and ATOM(PO) 
and we reach step E5. Step E5 sets BLINK (PO) <— A, etc., and a similar argument 
shows that we will ultimately mark all nodes on paths that start with BLINK (PO) and 
do not pass through PO or nodes reachable from ALINK(PO). Then we will get to E6 
with T = PO, P = BLINK (PO) , and finally we get to E6 with T = A, P = PO. 

4. The program that follows incorporates the suggested improvements in the speed 
of processing atoms that appear in the text after the statement of Algorithm E. 

In steps E4 and E5 of the algorithm, we want to test if MARK(Q) = 0. If NODE(Q) = 
+0, this is an unusual case that can be handled properly by setting it to —0 and treating 
it as if it were originally -0, since it has ALINK and BLINK both A. This simplification 
is not reflected in the timing calculations below. 

rll = P. rI2 = T, rI3 = Q, and rX = —1 (for setting MARKs). 

01 MARK EQU 0:0 

02 ATOM EQU 1:1 

03 ALINK EQU 2:3 

04 BLINK EQU 4:5 
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05 

El 

LD1 

P0 

1 

06 


ENT2 

0 

1 

07 


ENTX 

-1 

1 

08 

E2 

STX 

0,1 (MARK) 

1 

09 

E3 

LDA 

0,1 (ATOM) 

1 

10 


JAZ 

E4 

1 

11 

E6 

J2Z 

DONE 

n 

12 


ENT3 

0,2 

n — 1 

13 


LDA 

0,3 (ATOM) 

n — 1 

u 


JANZ 

IF 

n — 1 

15 


LD2 

0,3 (BLINK) 

£2 

16 


ST1 

0,3 (BLINK) 

£2 

17 


ENT1 

0,3 

t2 

18 


JMP 

E6 

£2 

19 

1H 

STZ 

0,2 (ATOM) 


20 


LD2 

0,3 (ALINK) 

£1 

21 


ST1 

0,3 (ALINK) 

£1 

22 


ENT1 

0,3 

£1 

23 

E5 

LD3 

0,1 (BLINK) 

n 

24 


J3Z 

E6 

n 

25 


LDA 

0,3 

n — b 2 

26 


STX 

0,3 (MARK) 

n — 62 

27 


JANP 

E6 

n — 62 

28 


LDA 

0,3 (ATOM) 

£2 + 0,2 

29 


JANZ 

E6 

£2 4- cl 2 

30 


ST2 

0,1 (BLINK) 

£2 

31 

E4A 

ENT2 

0,1 

n — 1 

32 


ENT1 

0,3 

n — 1 

33 

E4 

LD3 

0,1 (ALINK) 

n 

34 


J3Z 

E5 

n 

35 


LDA 

0,3 

n — 61 

36 


STX 

0,3 (MARK) 

n — 61 

37 


JANP 

E5 

n — bi 

38 


LDA 

0,3 (ATOM) 


39 


JANZ 

E5 

tl 4“ CLl 

40 


STX 

0,1 (ATOM) 

t\ 

41 


ST2 

0,1 (ALINK) 


42 


JMP 

E4A 

ti 


El. Initialize. P 4— P0. 

T 4- A. 
rX 4- -1. 

E2. Mark. MARK (P) 4 - 1. 

E3. Atom? 

Jump if ATOM(P) = 0. 

E6. Up, 

Q <- T. 

Jump if ATOM(T) = 1. 

T 4- BLINK (q). 

BLINK(Q) 4- P. 

P 4- Q. 

ATOM(T) 4- 0. 

T <- ALINK(Q). 

ALINK(Q) 4- P. 

P 4- Q. 

E5. Down BLINK. Q 4— BLINK (P). 
Jump if Q = A. 

MARK(q) 4- 1. 

Jump if NODE(q) was already marked. 

Jump if ATOM(Q) = 1. 

BLINK (P) 4- T. 

T 4- P. 

P «- Q- 

E4. Down ALINK. Q 4— ALINK(P) . 
Jump if Q = A. 

MARK(Q) 4- 1. 

Jump if NODE(q) was already marked. 

Jump if ATOM(q) = 1. 

ATOM (P) 4- 1. 

ALINK (P) 4- T. 

T 4- P, P 4 - q, to E4. | 


By Kirchhoff ’s law, ti + 1 2 + 1 = n. The total time is (34n + 4fi + 3a - 56 - 8 )«, when 
ii is the number of nonatomic nodes marked, a is the number of atoms marked, b i; 
the number of A links encountered in marked nonatomic nodes, and h is the numbe 
of times we went down an ALINK (0 < t x < n). 

5. (The following is the fastest known marking algorithm for a one-level memory.) 

51. Set MARK(PO) 4— 1. If ATOM(PO) = 1, the algorithm terminates; otherwise sei 
S 4— 0, R 4— PO, T 4— A. 

52. Set P 4 - BLINK (R) . If P = A or MARK(P) = 1 , go to S3. Otherwise sei 
MARK(P) 4— 1. Now if AT0M(P) = 1, go to S3; otherwise if S < N set S 4 — S + 1 
STACK [S] 4— P, and go to S3; otherwise go to S5. 
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53. Set P 4— ALINK(R). If P = A or MARK(P) = 1, go to S4. Otherwise set 
MARK(P) 4— 1. Now if ATOH(P) = 1, go to S4; otherwise set R 4— P and return 
to S2. 

54. If S = 0, terminate the algorithm; otherwise set R 4— STACK [S] , S 4— S — 1, 
and go to S2. 

55. Set Q 4— ALINK(P). If Q = A or MARK(Q) = 1, go to S6. Otherwise set 
MARK(Q) 4- 1. Now if ATOM(Q) = 1, go to S6; otherwise set ATQM(P) 4- 1, 
ALINK(P) 4— T, T 4- P, P 4- Q, go to S5. 

56. Set Q 4— BLINK (P). If Q = A or MARK(q) = 1, go to S7; otherwise set 
MARK(Q) 4- 1. Now if ATOM(Q) = 1, go to S7; otherwise set BLINK (P) 4- T, 
T 4— P, P 4— Q, go to S5. 

57. If T = A, go to S3. Otherwise set Q 4— T. If ATOM(q) = 1, set ATOM(q) 4— 0, 

T 4- ALINK(q), ALINK(Q) 4- P, P 4- Q, and return to S6. If ATOM(Q) = 0, set 
T 4- BLINK (Q) , BLINK (Q) <- P, P <- Q, and return to S7. | 

Reference: CACM 10 (1967), 501-506. 

6. From the second phase of garbage collection (or perhaps also the initial phase, 
if all mark bits are set to zero at that time). 

7. Delete steps E2 and E3, and delete “AT0M(P) 4— 1” in E4. Set MARK(P) 4— 1 in 
step E5 and use “MARK(Q) = 0”, “MARK(Q) = 1” in step E6 in place of the present 
l 'AT0M(Q) = 1”, “AT0M(Q) = 0” respectively. The idea is to set the MARK bit only after 
the left subtree has been marked. This algorithm works even if the tree has overlapping 
(shared) subtrees, but it does not work for all recursive List structures such as those 
with N0DE(ALINK(q) ) an ancestor of N0DE(Q). (Note that ALINK of a marked node is 
never changed.) 

8. Solution 1: Analogous to Algorithm E, but simpler. 

FI. Set T 4- A, P 4- P0. 

F2. Set MARK(P) 4- 1, and set P 4- P + SIZE(P) . 

F3. If MARK(P) = 1, go to F5. 

F4. Set Q 4- LINK(P) . If q ^ A and MARK(q) = 0, set LINK(P) 4- T, T 4- P, P 4- q 
and go to F2. Otherwise set P 4— P — 1 and return to F3. 

F5. If T = A, stop. Otherwise set q 4- T, T 4- LINK(q) , LINK(q) 4- P, P 4- q - 1, 
and return to F3. | 

A similar algorithm, which sometimes decreases the storage overhead and which 
avoids all pointers into the middle of nodes, has been suggested by Lars-Erik Thorelli, 
BIT 12 (1972), 555-568. 

Solution 2: Analogous to Algorithm D. For this solution, we assume that the SIZE 
field is large enough to contain a link address. Such an assumption is probably not 
justified by the statement of the problem, but it lets us use a slightly faster method 
than the first solution when it is applicable. 

Gl. Set T 4- A, MARK(PO) 4- 1, P 4- PO + SIZE(PO) . 

G2. If MARK(P) = 1, go to G5. 

G3. Set q 4- LINK(P) , P 4- P - 1. 

G4. If q / A and MARK(q) = 0, set MARK(q) 4- 1, S 4- SIZE(q), SIZE(q) 4- T, 
T 4- q + S. Go back to G2. 


■ 
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G5. If T = A, stop. Otherwise set P 4 — T and find the first value of Q = P, P - 1, 
P - 2, . . . for which MARK(Q) = 1; set T 4- SIZE(Q) and SIZE(Q) <- P - Q. Go 
back to G2. | 

9. HI. Set L 4- 0, K 4- M + 1, MARK(O) 4 - 1, MARK (M + 1)4-0. 

H2. Increase L by one, and if MARK(L) = 1 repeat this step. 

H3. Decrease K by one, and if MARK (K) = 0 repeat this step. 

H4. If L > K, go to step H5; otherwise set NODE(L) 4 - NODE(K) , ALINK(K) 4 - L. 
MARK(K) 4— 0, and return to H2. 

H5. For L = 1, 2, . . . , K do the following: Set MARK(L) 4 - 0. If ATOM(L) = 0 
and ALINK(L) > K, set ALINK(L) 4- ALINK(ALINK(L) ) . If ATOM(L) = 0 and 
BLINK (L) > K, set BLINK (L) 4 - ALINK (BLINK (L) ) . | 

See also exercise 2.5-33. 

10. Zl. [Initialize.] Set F 4- P0, R 4= AVAIL, N0DE(R) 4 - N0DE(F) , REF(F) 4 - R. (Here 
F and R are pointers for a queue set up in the REF fields of all header nodes 
encountered.) 

Z2. [Begin new List.] Set P 4- F, Q 4 - REF(P) . 

Z3. [Advance to right.] Set P 4- RLINK(P) . If P = A, go to Z 6 . 

Z4. [Copy one node.] Set Q1 4= AVAIL, RLINK(Q) 4 — Ql, Q 4 — Q1 N0DE(Q) 4 - 
N0DE(P) . 


Z5. [Translate sub-List link.] IfT(P) = 1, set PI 4- REF (P) , and if REF (PI) = A set 
REF(R) 4- PI, R 4= AVAIL. REF(Pl) 4- R, N0DE(R) 4- N0DE(P1), REF(Q) 4- R. 
If T(P) = 1 and REF(Pl) ^ A, set REF(Q) 4- REF(Pl). Go to Z3. 

Z6. [Move to next List.] Set RLINK(Q) 4- A. If REF (F) / R, set F 4 - REF (REF (F) ) 
and return to Z2. Otherwise set REF(R) 4 - A, P 4 - PO. 

Z7. [Final cleanup.] Set Q 4- REF(P). If Q / A, set REF (P) 4 — A and P 4 - Q and 
repeat step Z7. | 


Of course, this use of the REF fields makes it impossible to do garbage collection with 
Algorithm D; moreover, Algorithm D is ruled out by the fact that the Lists aren’t 
well-formed during the copying. 

Several elegant List-moving and List-copying algorithms that make substantially 
weaker assumptions about List representation have been devised See D W Clark 
CACM 19 (1976), 352-354; J. M. Robson, CACM 20 (1977), 431-433. 


11. Here is a pencil-and-paper method that can be written out more formally to answer 
the problem: First attach a unique name (e.g., a capital letter) to each List in the given 
set; in the example we might have A = ( a:C,b,a:F ), F = (b:D), B = (a: F,b,a: E), 
C — (b:G), G = ( a:C ), D = ( a: F ), E = (b:G). Now make a list of pairs of List 
names that must be proved equal. Successively add pairs to this list until either a. 
contradiction is found because we have a pair that disagree on the first level (then the 
originally given Lists are unequal), or until the list of pairs does not imply any further 
pairs (then the originally given Lists are equal). In the example, this list of pairs would 
originally contain only the given pair, AB\ then it gets the further pairs CF, EF (by 
matching A and B), DG (from CF); and then we have a self-consistent set. 

To prove the validity of this method, observe that (i) if it returns the answer 
“unequal”, the given Lists are unequal; (ii) if the given Lists are unequal, it returns 
the answer “unequal”; (iii) it always terminates. 
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12. When the AVAIL list contains N nodes, where N is a specified constant to be 
chosen as discussed below, initiate another coroutine that shares computer time with 
the main routine and does the following: (a) Marks all N nodes on the AVAIL list; (b) 
marks all other nodes that are accessible to the program; (c) links all unmarked nodes 
together to prepare a new AVAIL list for use when the current AVAIL list is empty, and 
(d) resets the mark bits in all nodes. One must choose N and the ratio of time sharing 
so that operations (a), (b), (c), and (d) are guaranteed to be complete before N nodes 
are taken from the AVAIL list, yet the main routine is running sufficiently fast. It is 
necessary to use some care in step (b) to make sure that all nodes “accessible to the 
program” are included, as the program continues to run; details are omitted here. If 
the list formed in (c) has fewer than N nodes, it may be necessary to stop eventually 
because memory space might become exhausted. [For further information, see Guy L. 
Steele Jr., CACM 18 (1975), 495-508; P. Wadler, CACM 19 (1976), 491-500; E. W. 
Dijkstra, L. Lamport, A. J. Martin, C. S. Scholten, and E. F. M. Steffens, CACM 21 
(1978), 966-975; H. G. Baker, Jr., CACM 21 (1978), 280-294.] 


SECTION 2.4 


1. Preorder. 


2. It is essentially proportional to the number of Data Table entries created. 

3. Change step A5 to: 

A5'. [Remove top level.] Remove the top stack entry; and if the new level number 
at the top of the stack is > L, let (LI, PI) be the new entry at the top of the 
stack and repeat this step. Otherwise set SIB (PI) <- Q and then let (LI, PI) 
be the new entry at the top of the stack. 

4. (Solution by David S. Wise.) Rule (c) is violated if and only if there is a data item 
whose complete qualification A 0 OF . . . OF A n is also a COBOL reference to some other 
data item. Since the parent A\ OF . . . OF A n must also satisfy rule (c), we may assume 
that this other data item is a descendant of the same parent. Therefore Algorithm A 
would be extended to check, as each new data item is added to the Data Table, whether 
its parent is an ancestor of any other item of the same name, or if the parent of any 
other item of the same name is in the stack. (When the parent is A, it is everybody’s 
ancestor and always on the stack.) 

On the other hand, if we leave Algorithm A as it stands, the COBOL programmer 
will get an error message from Algorithm B when trying to use an illegal item. Only 
MOVE CORRESPONDING can make use of such items without error. 


5. Make these changes: 


Step 

replace 

Bl. 

P <- LINK (Po) 

B2. 

k <r- 0 

B3. 

k < n 

B4. 

k k + 1 

B6. 

NAME(S) = P k 


by 

P <- LINK (INFO (T) ) 
K <— T 

RLINK(K) / A 
K t- RLINK(K) 
NAME(S) = INF0(K) 


6. A simple modification of Algorithm B makes it search only for complete references 
(if k = n and PARENT(S) ^ A in step B3, or if NAME (S) / P k in step B6, set P t— PREV(P) 
and go to B2). The idea is to run through this modified Algorithm B first; then, if Q is 
still A, to perform the unmodified algorithm. 

7 . MOVE MONTH OF DATE OF SALES TO MONTH OF DATE OF PURCHASES . MOVE DAY OF DATE 
OF SALES TO DAY OF DATE OF PURCHASES. MOVE YEAR OF DATE OF SALES TO YEAR OF 
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DATE OF PURCHASES. MOVE ITEM OF TRANSACTION OF SALES TO ITEM OF TRANSACTION OF 
PURCHASES. MOVE QUANTITY OF TRANSACTION OF SALES TO QUANTITY OF TRANSACTION 
OF PURCHASES. MOVE PRICE OF TRANSACTION OF SALES TO PRICE OF TRANSACTION OF 
PURCHASES. MOVE TAX OF TRANSACTION OF SALES TO TAX OF TRANSACTION OF PURCHASES. 

8. If and only if a or 0 is an elementary item. (It may be of interest to note that 
the author failed to handle this case properly in his first draft of Algorithm C, and it 
actually made the algorithm more complicated.) 

9. "MOVE CORRESPONDING a TO /3”, if neither a nor /3 is elementary, is equivalent to 
the set of statements "MOVE CORRESPONDING A OF a TO A OF /3” taken over all names 
A common to groups a and /?. (This is a more elegant way to state the definition 
than the more traditional and more cumbersome definition of "MOVE CORRESPONDING” 
given in the text.) We may verify that Algorithm C satisfies this definition, using an 
inductive proof that steps C2 through C5 will ultimately terminate with P = PO and 
Q = QO. Further details of the proof are filled in as we have done many times before in 
a “tree induction” (see, for example, the proof of Algorithm 2. 3. IT). 

10. (a) Set SI * LINK ( P k ) . Then repeatedly set SI t— PREV(Sl) zero or more times 
until either SI = A (NAME(S) / P k ) or SI = S (NAME(S) = P k ). (b) Set PI <- P and 
then set PI t— PREV(Pl) zero or more times until PREV(Pl) = A; do a similar operation 
with variables Q1 and Q; then test if PI = Ql. Alternatively, if the Data Table entries 
are ordered so that PREV(P) < P for all P, a faster test can be made in an obvious way 
depending on whether P > Q or not, following the PREV links of the larger to see if the 
smaller is encountered. 

11. A minuscule improvement in the speed of step C4 would be achieved by adding a 
new link field SIB1(P) = CHILD (PARENT(P) ) . More significantly, we could modify the 
CHILD and SIB links so that NAME(SIB(P)) > NAME(P); this would speed up the search 
in step C3 considerably because it would require only one pass over each family to find 
the matching members. This change would therefore remove the only “search” present 
in Algorithm C. Algorithms A and C are readily modified for this interpretation, and 
the reader may find it an interesting exercise. (However, if we consider the relative 
frequency of MOVE CORRESPONDING statements and the usual size of family groups, the 
resulting speedup will not be terribly significant in the translation of actual COBOL 
programs.) 

12. Leave steps Bl, B2, B3 unchanged; change the other steps thus: 

B4. Set k <- k+ 1, R «— LINK (P*. ) . 

B5. If R = A, set P t— PREV(P) and go to B2 (we haven’t found a match). If 
R < S < SCOPE(R) , set S <— R and go to B3. Otherwise set R PREV(R) and 
repeat step B5. | 

This algorithm does not adapt to the PL/I convention of exercise 6. 

13. Use the same algorithm, minus the operations that set NAME, PARENT, CHILD, 
and SIB. Whenever removing the top stack entry in step A5, set SCOPE(Pl) t- Q - 1. 
When the input is exhausted in step A2, simply set L t— 0 and continue, then terminate 
the algorithm if L = 0 in step A7. 

14. The following algorithm, using an auxiliary stack, has steps numbered to show a 
direct correspondence with the text’s algorithm. 

Cl. Set P <— PO, Q <— QO, and set the stack contents empty. 
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C2. If SCOPE(P) — P or SCOPE(Q) = Q, output (P,Q) as one of the desired pairs 
and go to C5. Otherwise put (P, Q) on the stack and set P 4 - P + 1, Q 4 - Q + 1. 

C3. Determine if P and Q point to entries with the same name (see exercise 10(b)). 
If so, go to C2. If not, let (P1,Q1) be the entry at the top of the stack; if 
SCOPE(Q) < SCOPE(Ql), set Q 4— SCOPE(Q) + 1 and repeat step C3. 

C4. Let (PI, Ql) be the entry at the top of the stack. If SCOPE(P) < SCOPE(Pl), set 
P 4- SCOPE(P) + 1, Q 4- qi + 1, and go back to C3. If SCOPE(P) = SCOPE(Pl), 
set P 4— PI, Q 4— Ql and remove the top entry of the stack. 

C5. If the stack is empty, the algorithm terminates. Otherwise go to C4. | 

SECTION 2.5 

1. In such fortuitous circumstances, a stack-like operation may be used as follows: 
Let the memory pool area be locations 0 through M - 1, and let AVAIL point to the 
lowest free location. To reserve N words, report failure if AVAIL + N > M, otherwise set 
AVAIL 4— AVAIL + N. To free these N words, just set AVAIL 4— AVAIL — N. 

Similarly, cyclic queue-like operation is appropriate for a first-in-first-out discipline. 

2 . The amount of storage space for an item of length l is k \l/(k - 6)] , which has the 
average value kL/(k — b) + (1 — a)k, where a is assumed to be 1/2, independent of k. 
This expression is a minimum (for real values of k) when k = b + V2bL. So choose k to 
be the integer just above or just below this value, whichever gives the lowest value of 
kL/ (k - b) + \k. For example, if b = 1 and L = 10, we would choose k « 1 + \/20 = 5 
or 6; both are equally good. For much greater detail about this problem, see JACM 
12 (1965), 53-70. 

4. rll = Q, rI2 = P. 

A1 LDA N rA 4— N. 

ENT2 AVAIL P 4- LOC(AVAIL) . 

A2A ENT1 0,2 Qf-P. 

A2 LD2 0,1 (LINK) P 4— LINK(Q) . 

J2N OVERFLOW If P = A, no room. 

A3 CMPA 0,2 (SIZE) 

JG A2A Jump if N > SIZE(P). 

A4 SUB 0,2(SIZE) rA 4- N — SIZE(P) = K. 

JANZ *+3 Jump if K / 0. 

LDX 0,2 (LINK) 

STX 0 , 1 (LINK) LINK(Q) 4— LINK(P). 

STA 0,2(SIZE) SIZE(P) 4— K. 

LD1 0,2(SIZE) Optional ending, 

INC1 0,2 sets rll 4- P + K. | 

5. Probably not. The unavailable storage area just before location P will subsequently 
become available, and its length will be increased by the amount K; an increase of 99 
would not be negligible. 

6. The idea is to try to search in different parts of the AVAIL list each time. We can 
use a “roving pointer,” called ROVER for example, which is treated as follows: In step 
Al, set Q 4- ROVER. After step A4, set ROVER 4- LINK(Q) if LINK(Q) / A, otherwise 
set ROVER 4— L0C (AVAIL). In step A2, when P = A the first time during a particular 
execution of Algorithm A, set Q 4— LOC(AVAIL) and repeat step A2. When P = A the 
second time, the algorithm terminates unsuccessfully. In this way ROVER will tend to 
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point to a random spot in the AVAIL list, and the sizes will be more balanced. At the 
beginning of the program, set ROVER 4— LOC (AVAIL) ; it is also necessary to set ROVER to 
LOC (AVAIL) everywhere else in the program where the block whose address equals the 
current setting of ROVER is taken out of the AVAIL list. (Sometimes, however, it is useful 
to have small blocks at the beginning, as in the strict first-fit method; for example, we 
might want to keep a sequential stack at the high end of memory. In such cases we can 
reduce the search time by using trees as suggested in exercise 6.2.3-30.) 

7. 2000, 1000 with requests of sizes 800, 1300. [An example where worst-fit succeeds, 
while best-fit fails, has been constructed by R. J. Weiland.] 

8. In step Al, also set M 4— oo, R 4— A. In step A2, if P = A go to A6. In step A3, go 
to A5 instead of A4. Add new steps as follows: 

A5. [Better fit?] If M > SIZE(P), set R 4- Q and M 4- SIZE(P). Then set Q 4- P 
and return to A2. 

A6. [Any found?] If R = A, the algorithm terminates unsuccessfully. Otherwise 
set Q 4— R, P 4— LINK(Q), and go to A4. | 

9. Obviously if we are so lucky as to find SIZE(P) = N, we have a best fit and it is not 
necessary to search farther. (When there are only very few different block sizes, this 
occurs rather often.) If a “boundary tag” method like Algorithm C is being used, it is 
possible to maintain the AVAIL list in sorted order by size; so the length of search could 
be cut down to half the length of the list or less, on the average. But the best solution 
is to make the AVAIL list into a balanced tree structure as described in Section 6.2.3, if 
it is expected to be long. 

10. Make the following changes: 

Step B2, for “P > P0” read “P > P0”. 

At the beginning of step B3, insert “If P0 + N > P and P / A, set N 4— max(N, 
P + SIZE(P) — P0), P 4— LINK(P), and repeat step B3.” 

Step B4, for “Q + SIZE(Q) = PO”, read “Q + SIZE(Q) > P0”; and for “SIZE(Q) 4- 
SIZE(Q) +N” read “SIZE(Q) 4— max(SIZE(Q) , PO + N — Q)”. 

11. If P0 is greater than ROVER, we can set Q 4— ROVER instead of Q 4- LOC (AVAIL) in 
step Bl. If there are n entries in the AVAIL list, the average number of iterations of 
step B2 is (2n + 3)(n + 2)/(6n + 6) = |n + | + O (^). For example if n = 2 we get 9 
equally probable situations, where PI and P2 point to the two existing available blocks: 


ROVER = PI 
ROVER = P2 
ROVER = LOC (AVAIL) 


PO < PI PI < PO < P2 P2 < P0 

1 1 1 2 

1 2 | 1 
1 2 3 


This chart shows the number of iterations needed in each case. The average is 




14 

IT 


12. Al. Set P 4- ROVER, F 4- 0. 

A2. If P = LOC(AVAIL) and F = 0, set P 4— AVAIL, F <— 1, and repeat step A2. If 
P = LOC (AVAIL) and F ^ 0, the algorithm terminates unsuccessfully. 

A3. If SIZE(P) > N, go to A4; otherwise set P 4— LINK(P) and return to A2. 
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A4. Set ROVER 4 — LINK(P), K 4— SIZE(P) — N. If K < c (where c is a constant > 2), 
set LINK (LINK (P + 1)) 4- ROVER, LINK (ROVER + 1) 4 - LINK(P + 1), L 4 - P; 
otherwise set L 4— P + K, SIZE(P) 4- SIZE(L — 1) 4— K, TAG(L - 1) 4 - 
SIZE(L) 4- N. Finally set TAG(L) 4- TAG(L + SIZE(L) - 1) 4 - | 


rll = 

P, rX 

III 

y 

to 

III 

p 


LINK 

EQU 

4:5 


SIZE 

EQU 

1:2 


TSIZE 

EQU 

0:2 


TAG 

EQU 

0:0 


A1 

LDA 

N 

rA 1- N. 


SLA 

3 

Shift into SIZE field. 


ENTX 

0 

F 4- 0. 


LD1 

ROVER 

P 4- ROVER. 


JMP 

A2 


A3 

CMPA 

0, 1 (SIZE) 



JLE 

A4 

Jump if N < SIZE(P) . 


LD1 

0,1 (LINK) 

P 4- LINK(P) . 

A2 

ENT2 

-AVAIL, 1 

rI2 4- P - L0C( AVAIL). 


J2NZ 

A3 



JXNZ 

OVERFLOW 

Is F / 0? 


ENTX 

1 

Set F 4— 1. 


LD1 

AVAIL(LINK) 

P 4- AVAIL. 


JMP 

A2 


A4 

LD2 

0,1 (LINK) 



ST2 

ROVER 

ROVER 4- LINK(P) . 


LDA 

0,1(SIZE) 

rA = K 4- SIZE(P) - N. 


SUB 

N 



CMPA 

=c= 



JGE 

IF 

Jump if K > c. 


LD3 

1,1 (LINK) 

rI3 4- LINK(P + 1). 


ST2 

0,3(LINK) 

LINK(rI3) 4- ROVER. 


ST3 

1,2 (LINK) 

LINK (ROVER + 1) 4- rI3. 


ENT 2 

0,1 

L 4-P. 


LD3 

0 , 1 (SIZE) 

rI3 4- SIZE(P) . 


JMP 

2F 


1H 

STA 

0 , 1 (SIZE) 

SIZE(P) <r- K. 


LD2 

0, 1 (SIZE) 



INC2 

0,1 

L <- P + K. 


LDAN 

0,1 (SIZE) 

rA 4 K. 


STA 

-1 ,2 (TSIZE) 

SIZE(L — 1) 4— K, TAG(L - 1) <- 


LD3 

N 

rI3 4- N. 

2H 

ST3 

0,2(TSIZE) 

TAG(L) 4- also set SIZE(L) 


INC3 

0,2 



STZ 

-1,3(TAG) 

TAG(L + SIZE(L) - 1) 4- “+”. 


14. (a) This field is needed to locate the beginning of the block, in step C2. It could 
be replaced (perhaps to advantage) by a link to the first word of the block. See also 
exercise 19. (b) This field is needed because we sometimes need to reserve more than N 
words (for example if K = 1), and the amount reserved must be known when the block 
is subsequently freed. 
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15, 16. rll = PO, rI2 = PI, rI3 = F, rI4 = B, rI6 = -N. 


D1 

LD1 

PO 

Dl. 


LD2 

0 , 1 (SIZE) 



ENN6 

0,2 

N 4- SIZE(PO) . 


INC2 

0,1 

PI 4- P0 + N. 


LD5 

0,2(TSIZE) 



J5N 

D4 

To D4 if TAG(Pl) = 

D2 

LD5 

-1 , 1 (TSIZE) 

D2. 


J5N 

D7 

To D7 if TAG(P0 - 1) = 

D3 

LD3 

AVAIL(LINK) 

D3. Set F 4- AVAIL. 


ENT4 

AVAIL 

B 4- L0C( AVAIL). 


JMP 

D5 

To D5. 

D4 

INC6 

0,5 

D4. N 4— N + SIZE(Pl) . 


LD3 

0,2 (LINK) 

F 4- LINK(Pl). 


LD4 

1,2 (LINK) 

B 4- LINK(P1 + 1). 


CMP 2 

ROVER 

(New code, because of the ROVER 


JNE 

*+3 

feature of exercise 12: 


ENTX 

AVAIL 

If PI = ROVER, 


STX 

ROVER 

set ROVER 4- L0C (AVAIL) .) 


DEC2 

0,5 

PI 4- PI + SIZE(Pl). 


LD5 

-1,1 (TSIZE) 



J5N 

D6 

To D6 if TAG(P0 - 1) = 

D5 

ST3 

0,1 (LINK) 

D5. LINK(PO) 4- F. 


ST4 

1,1 (LINK) 

LINK(P0 + 1) 4- B. 


ST1 

1 ,3(LINK) 

LINK(F + 1) 4- P0. 


ST1 

0,4 (LINK) 

LINK(B) 4- P0. 


JMP 

D8 

To D8. 

D6 

ST3 

0,4(LINK) 

D6. LINK(B) 4- F. 


ST4 

1 ,3(LINK) 

LINK(F + 1) 4- B. 

D7 

INC6 

0,5 

D7. N 4 — N T SIZE(P0 - 1). 


INC1 

0,5 

P0 4- PO - SIZE(P0 - 1). 

D8 

ST6 

0,1 (TSIZE) 

D8. SIZE(PO) 4- N. TAG(PO) 4- 


ST6 

-1 , 2 (TSIZE) 

SIZE(P1 - 1) 4- N, TAG(P1 - 1) 4- 


17. Both LINK fields equal to LOC(AVAIL). 

18. Algorithm A reserves the upper end of a large block. When storage is completely 
available, the first-fit method actually begins by reserving the high-order locations, but 
once these become available again they are not re-reserved since a fit is usually found 
already in the lower locations; thus the initial large block at the lower end of memory 
quickly disappears with first-fit. A large block rarely is the best fit, however, so the 
best-fit method leaves a large block at the beginning of memory. 

19. Use the algorithm of exercise 12, except delete the references to SIZE(L - 1). 
TAG(L — 1) , and TAG(L + SIZE(L) — 1) from step A4; also insert the following new step 
between steps A2 and A3: 

A2~. Set PI 4— P + SIZE(P) . IfTAG(Pl) = “+”, proceed to step A3. Otherwise set 
P2 4- LINK(Pl), LINKCP2 + 1) 4- LINK (PI + 1), LINK(LINK(P1 + 1)) 4- P2. 
SIZE(P) 4- SIZE(P) + SIZE(Pl). If ROVER = PI, set ROVER 4- P2. Repeat 
step A2|. 
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Clearly the situation of (2), (3), (4) can’t occur here; the only real effect on 
storage allocation is that the search here will tend to be longer than in exercise 12, 
and sometimes K will be less than c although there is really another available block 
preceding this one that we do not know about. 

(An alternative is to take the collapsing out of the inner loop A3, and to do the 
collapsing only in step A4 before the final allocation or in the inner loop when the 
algorithm would otherwise have terminated unsuccessfully. This alternative requires a 
simulation study to see if it is an improvement or not.) 

[This method, with a few refinements, has proved to be quite satisfactory in the 
implementations of Tf;X and METflFONT. See TjrjK: The Program (Addison— Wesley, 
1986), §125.] 

20. When a buddy is found to be available, during the collapsing loop, we want to 
remove that block from its AVAIL [fc] list, but we do not know which links to update 
unless (i) we do a possibly long search, or (ii) the list is doubly linked. 

21 . If n = 2 k a, where 1 < a < 2, a n is 2 2k+1 (a - §) + ±, and b n is 2 2k ~ l a 2 + 2 k ~ 1 a. 
The ratio a n /b n for large n is essentially 4(a — |)/a 2 , which takes its minimum value 
3 when a = 1 and 2, and its maximum value § when a = l|. So a n /b n approaches no 
limit; it oscillates between these two extremes. The averaging methods of Section 4.2.4 
do, however, yield an average ratio of 4(ln2) _1 f*(a - § )da/a 3 = (ln2) _1 « 1.44. 

22. This idea requires a TAG field in several words of the 11-word block, not only in 
the first word. It is a workable idea, if those extra TAG bits can be spared, and it would 
appear to be especially suitable for use in computer hardware. 

23. 011011110100; 011011100000. 

24. This would introduce a bug in the program; we may get to step SI when TAG(0) = 1, 
since S2 may return to SI. To make it work, add “TAG(L) 0” after “L <- P” in step S2. 
(it is easier to assume instead that TAG(2 m ) = 0.) 

25. The idea is absolutely correct. (Criticism need not be negative.) The list heads 
AVAIL [fc] may be eliminated for n < k < to; the algorithms of the text may be used if 
“m” is changed to “n” in steps Rl, SI. The initial conditions (13) and (14) should be 
changed to indicate 2 m ~ n blocks of size 2 n instead of one block of size 2 m . 

26. Using the binary representation of M, we can easily modify the initial conditions 
Us): (14) so that all memory locations are divided into blocks whose size is a power of 
two, with blocks in decreasing order of size. In Algorithm S, TAG(P) should be regarded 
as 0 whenever P > M — 2 k . 

27. rll = k, rI2 = j, rI3 = j - k, rI4 = L, L0C(AVAIL[j] ) = AVAIL + j; as- 

sume that there is an auxiliary table TW0[j] = 2 J , stored in location TWO + j, for 
0 < j < m. Assume further that “+” and represent tags of 0 and 1, and that 
TAG(L0C(AVAIL[j] ) ) = ”; but TAG (L0C (AVAIL [rri + 1] ) ) = f " is a sentinel. 

00 KVAL EQU 5 : 5 

01 TAG EQU 0:0 

02 LINKF EQU 1:2 

03 LINKB EQU 3:4 

04 TLNKF EQU 0:2 

05 Rl LD1 K 1 Rl. Find block. 

06 ENT2 0,1 1 j <■ k. 

07 ENT3 0 1 

08 LD4 AVAIL, 2 (LINKF) 1 
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09 

1H 

ENT 5 

AVAIL ,2 

1 + R 


10 


DEC5 

0,4 

1 + jR 


11 


J5NZ 

R2 

1 + R 

Jump if AVAILFLj] / LOC (AVAIL [j] ) . 

12 


INC2 

1 

R 

Increase j. 

13 


INC3 

1 

R 


U 


LD4N 

AVAIL,2(TLNKF) 

R 


15 


J4NN 

IB 

R 

Is j < m? 

16 


JMP 

OVERFLOW 



17 

R2 

LD5 

0,4(LINKB) 

1 

R2. Remove from list. 

18 


ST5 

AVAIL,2(LINKB) 

1 

AVAILB [j] 4- LINKB (L). 

19 


ENTA 

AVAIL, 2 

1 


20 


STA 

0,5(LINKF) 

1 

LINKF(L) 4- LOC ( AVAIL [j] ) . 

21 


STZ 

0,4(TAG) 

1 

TAG(L) 4- 0. 

22 

R3 

J3Z 

DONE 

1 

R3. Split reauired? 

23 

R4 

DEC3 

1 

R 

R4. Split. 

24 


DEC2 

1 

R 

Decrease j. 

25 


LD5 

TWO, 2 

R 

rI5 = P. 

26 


INC5 

0,4 

R 

P 4- L + 2+ 

27 


ENNA 

AVAIL, 2 

R 


28 


STA 

0,5(TLNKF) 

R 

TAG(P) 4- 1, LINKF(P) 4- LOC (AVAIL [j] ) 

29 


STA 

0,5(LINKB) 

R 

LINKB(P) 4- LOC (AVAIL [j]). 

30 


ST5 

AVAIL,2(LINKF) 

R 

AVAILF [j] 4-P. 

31 


ST5 

AVAIL,2(LINKB) 

R 

AVAILB [j] 4-P. 

32 


ST2 

0,5(KVAL) 

R 

KVAL(P) 4 - j. 

S3 


J3P 

R4 

R 

Go to R3. 

34 

DONE 




1 

28. 

rll = 

k, rI5 

= P, rI4 = L; assume TAG (2 

i m ^ 


01 

SI 

LD4 

L 

1 

SI. Is buddy available? 

02 


LD1 

K 

1 


03 

1H 

ENTA 

0,4 

1 + S 


04 


XOR 

TWO, 1 

1 + S 

rA i — buddy*; (L) . 

05 


STA 

TEMP 

1 + S 


06 


LD5 

TEMP 

1 + S 

P 4- rA. 

07 


LDA 

0,5 

1 + S 


08 


JANN 

S3 

1 + S 

Jump if TAG(P) = 0. 

09 


CMP1 

0,5(KVAL) 

B + S 


10 


JNE 

S3 

B + S 

Jump if KVAL(P) / k. 

11 

S2 

LD2 

0,5(LINKB) 

S 

S2. Combine with buddv. 

12 


LD3 

0,5(LINKF) 

S 


13 


ST3 

0,2(LINKF) 

S 

LINKF (LINKB (P) ) 4-LINKF(P). 

14 


ST2 

0,3(LINKB) 

S 

LINKB(LINKF (P) ) 4— LINKB (P) . 

15 


INC1 

1 

S 

Increase k. 

16 


CMP4 

TEMP 

S 


17 


JL 

IB 

S 


18 


ENT4 

0,5 

A 

If L > P, set L 4— P. 

19 


JMP 

IB 

A 


20 

S3 

LD2 

AVAIL, l(LINKF) 

1 

S3. Put on list. 

21 


ENNA 

AVAIL, 1 

1 


22 


STA 

0, 4(0:4) 

1 

TAG(L) 4- 1, LINKB (L) 4- LOC (AVAIL [fc] ) 
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23 

ST2 

0,4(LINKF) 

1 

LINKF(L) 4- AVAILFM. 

24 

ST1 

0,4(KVAL) 

1 

KVAL(L) 4- k. 

25 

ST4 

0,2(LINKB) 

1 

LINKB ( AVAILF [fc] ) 4- L. 

26 

ST4 

AVAIL .l(LINKF) 

1 

AVAIL [fc] 4— L. | 


29. Yes, but only at the expense of some searching, or (better) an additional table 
of TAG bits packed somehow. (It is tempting to suggest that buddies not be joined 
together during Algorithm S, but only in Algorithm R if there is no block large enough 
to meet the request; but that would probably lead to a badly fragmented memory.) 

31. See David L. Russell, SICOMP 6 (1977), 607-621. 

32. Steven Crain points out that the method always frees all blocks and starts afresh 
before 16667 units of time have elapsed; hence the stated limit certainly exists. Proof: 
Let u n =n + t n , so that g n = |_§ min(10000, f(u n -i -n), f{u n - 2 -n), . . . , /(u 0 -n))j. 
Let Xo — 0 and x\ = uo, and Zfc+i = max(uo, ■ • • ,u Xk -i) for k > 1. If m*, > x k -i then 

„ , 5 ,, ,5 1 5 1 

U n < n + ~^f{xk - n) = -x k - -n < -x k - -Zfc-i for x k -\ < n < x k \ 

therefore x k +i — x k < \{x k — x k -i), and we must have x k = x k -i before reaching time 
12500 + [12500/4J + [12500/4 2 ] + • • • . 

33. Gl. [Clear LINKs.] Set P 4— 1, and repeat the operation LINK(P) 4— A, P 4— 

P + SIZE(P) until P = AVAIL. (This merely sets the LINK field in the first 
word of each node to A; we may assume in most cases that this step is 
unnecessary, since LINK(P) is set to A in step G9 below and it can be set to 
A by the storage allocator.) 

G2. [Initialize marking phase.] Set TOP 4- USE, LINK (TOP) 4- AVAIL, LINK (AVAIL) 
4— A. (TOP points to the top of a stack as in Algorithm 2.3.5D.) 

G3. [Pop up stack.] Set P 4- TOP, TOP 4- LINK (TOP) . If TOP = A, go to G5. 

G4. [Put new links on stack.] For 1 < k < T(P), do the following operations: 

Set Q 4- LINK(P + k)\ then if Q A and LINK(Q) = A, set LINK(Q) 4- TOP, 
TOP 4— Q. Then go back to G3. 

G5. [Initialize next phase.] (Now P = AVAIL, and the marking phase has been 
completed so that the first word of each accessible node has a nonnull LINK. 
Our next goal is to combine adjacent inaccessible nodes, for speed in later 
steps, and to assign new addresses to the accessible nodes.) Set Q -< — 1, 
LINK(AVAIL) 4— Q, SIZE(AVAIL) 4— 0, P 4— 1. (Location AVAIL is being used 
as a sentinel to signify the end of a loop in subsequent phases.) 

G6. [Assign new addresses.] If LINK (P) = A, go to G7. Otherwise if SIZE(P) = 0, 
go to G8. Otherwise set LINK(P) 4- Q, q 4- Q + SIZE(P) , P 4- P + SIZE(P) , 
and repeat this step. 

G7. [Collapse available areas.] If LINK(P + SIZE(P) ) = A, increase SIZE(P) by 
SIZE(P + SIZE(P)) and repeat this step. Otherwise set P 4— P + SIZE(P) and 
return to G6. 

G8. [Translate all links.] (Now the LINK field in the first word of each accessible 
node contains the address to which the node will be moved.) Set USE 4— 
LINK (USE), and AVAIL 4- Q. Then set P 4— 1, and repeat the following 
operation until SIZE(P) = 0: If LINK (P) / A, set LINK(Q) 4- LINK (LINK (Q) ) 
for all Q such that P < Q < P + T(P) and LINK(Q) / A; then regardless of the 
value of LINK(P) , set P 4- P + SIZE(P). 
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G9. [Move.] Set P -f— 1, and repeat the following operation until SIZE(P) = 0: 
Set Q <- LINK(P), and if Q ^ A set LINK(P) <— A and NODE(Q) t— NODE(P): 
then whether Q = A or not, set P P + SIZE(P) . (The operation NODE(Q) «- 
NODE(P) implies the movement of SIZE(P) words; we always have Q < P, so it 
is safe to move the words in order from smallest location to largest.) | 

[This method is called the “LISP 2 garbage collector.” An interesting alternative, which 
does not require the LINK held at the beginning of a node, can be based on the idea 
of linking together all pointers that point to each node — see Lars-Erik Thorelli, BIT 
16 (1976), 426-441; R. B. K. Dewar and A. P. McCann, Software Practice & Exp. 
7 (1977), 95-113; F. Lockwood Morris, CACM 21 (1978), 662-665, 22 (1979), 571; 
H. B. M. Jonkers, Inf. Proc. Letters 9 (1979), 26-30; J. J. Martin, CACM 25 (1982), 
571-581; F. Lockwood Morris, Inf. Proc. Letters 15 (1982), 139-142, 16 (1983), 215. 
Other methods have been published by B. K. Haddon and W. M. Waite, Comp. J. 10 
(1967), 162-165; B. Wegbreit, Comp. J. 15 (1972), 204-208; D. A. Zave, Inf. Proc. 
Letters 3 (1975), 167-169. Cohen and Nicolau have analyzed four of these approaches 
in ACM Trans. Prog. Languages and Systems 5 (1983), 532-553.] 


34. 

Let TOP = 

rll, Q = rI2, 

T) 

III 

OC 

k = rI4, SIZE(P) = rI5. Assume further 

A = 

0, and LINK(O) / 0 to simplify step G4. Step G1 is omitted. 

01 

LINK 

EQU 

4:5 



02 

INFO 

EQU 

0:3 



03 

SIZE 

EQU 

1:2 



04 

T 

EQU 

3:3 



05 

G2 

LD1 

USE 

1 

G2. Initialize marking phase. TOP <— USE. 

06 


LD2 

AVAIL 

1 


07 


ST2 

0,1 (LINK) 

1 

LINK(TOP) -t- AVAIL. 

08 


STZ 

0,2 (LINK) 

1 

LINK (AVAIL) <- A. 

09 

G3 

ENT3 

0,1 

a T 1 

G3. Pop up stack. P ■(— TOP. 

10 


LD1 

0,1 (LINK) 

a T 1 

TOP LINK (TOP). 

11 


J1Z 

G5 

a + 1 

To G5 if TOP = A. 

12 

G4 

LD4 

0,3(T) 

a 

G4. Put new links on stack, k «— T(P). 

13 

1H 

J4Z 

G3 

a + b 

k = 0? 

14 


INC3 

1 

b 

P «- P + 1. 

15 


DEC4 

1 

b 

k <— k — 1. 

16 


LD2 

0,3 (LINK) 

b 

Q <r- LINK(P) . 

17 


LDA 

0,2 (LINK) 

b 


18 


JANZ 

IB 

b 

Jump if LINK(Q) / A. 

19 


ST1 

0,2 (LINK) 

a — 1 

Otherwise set LINK(Q) <— TOP, 

20 


ENT1 

0,2 

a — 1 

TOP •(- Q. 

21 


JMP 

IB 

a — 1 


22 

G5 

ENT2 

1 

1 

G5. Initialize next phase. Q 1. 

23 


ST2 

0,3 

1 

LINK(AVAIL) <- 1, SIZE (AVAIL) <- 0. 

24 


ENT3 

1 

1 

P<-1. 

25 


JMP 

G6 

1 


26 

1H 

ST2 

0,3(LINK) 

a 

LINK(P) <- Q. 

27 


INC2 

0,5 

a 

Q <- Q + SIZE(P) . 

28 


INC3 

0,5 

a 

P1-P + SIZE(P). 

29 

G6 

LDA 

0,3(LINK) 

a -f 1 

G6. Assign new addresses. 

30 

G6A 

LD5 

0,3(SIZE) 

a + c + 1 



ANSWERS TO EXERCISES 615 


2.5 


31 


JAZ 

G7 

a + c + 1 

Jump if LINK(P) = A. 

32 


J5NZ 

IB 

a T 1 

Jump if SIZE(P) / 0. 

33 

G8 

LD1 

USE 

1 

G8. Translate all links. 

34 


LDA 

0 , 1 (LINK) 

1 


35 


STA 

USE 

1 

USE +- LINK (USE). 

36 


ST2 

AVAIL 

1 

AVAIL +- Q. 

31 


ENT3 

1 

1 

P T- 1. 

38 


JMP 

G8P 

1 


39 

1H 

LD6 

0,6(SIZE) 

d 


40 


INC5 

0,6 

d 

rI5 •+- rI5 T SIZE(P T SIZE(P)). 

41 

G7 

ENT6 

0,3 

c + d 

G7. Collapse available areas. 

42 


INC6 

0,5 

c + d 

rI6 T- P T SIZE(P) . 

43 


LDA 

0,6 (LINK) 

c + d 


44 


JAZ 

IB 

c + d 

Jump if LINK(rI6) = A. 

45 


ST5 

0,3(SIZE) 

c 

SIZE(P) +- rI5. 

46 


INC3 

0,5 

c 

P <- P + SIZE(P) . 

47 


JMP 

G6A 

c 


48 

2H 

DEC4 

1 

b 

k +- k — 1. 

49 


INC2 

1 

b 

q + — q T l . 

50 


LD6 

0,2 (LINK) 

b 


51 


LDA 

0,6(LINK) 

b 


52 


STA 

0,2(LINK) 

b 

LINK(q) <— LINK(LINK(q) ) . 

53 

1H 

J4NZ 

2B 

a + b 

Jump if k ^ 0. 

54 

3H 

INC3 

0,5 

a + c 

P T- PT SIZE(P). 

55 

G8P 

LDA 

0,3 (LINK) 

1 + a + c 


56 


LD5 

0,3(SIZE) 

1 + a + c 


51 


JAZ 

3B 

1 + a + c 

Is LINK(P) = A? 

58 


LD4 

0,3(T) 

1 T a 

k <- T (P) . 

59 


ENT2 

0,3 

1 + a 

q P- 

60 


J5NZ 

IB 

IT a 

Jump unless SIZE(P) = 0. 

61 

G9 

ENT3 

1 

1 

G9. Move. P t- 1. 

62 


ENT1 

1 

1 

Set rll for MOVE instructions. 

63 


JMP 

G9P 

1 


64 

1H 

STZ 

0,3(LINK) 

a 

LINK(P) +- A. 

65 


ST5 

*+1(4:4) 

a 


66 


MOVE 

0,3(*) 

a 

NODE(rll) N0DE(P) , rll <- rll T SIZE(P) 

61 

3H 

INC3 

0,5 

a T c 

P<-P + SIZE(P) . 

68 

G9P 

LDA 

0,3 (LINK) 

1 T a T c 


69 


LD5 

0,3(SIZE) 

1 T a T c 


10 


JAZ 

3B 

1 T a T c 

Jump if LINK(P) = A. 

11 


J5NZ 

IB 

1 T a 

Jump unless SIZE(P) =0. | 


In line 66 we are assuming that the size of each node is sufficiently small that it can 
be moved with a single MOVE instruction; this seems a fair assumption for most cases 
when this kind of garbage collection is applicable. 

The total running time for this program is (44a + 17b + 2w + 25c + 8 d + 47) u, 
where a is the number of accessible nodes, b is the number of link fields therein, c is 
the number of inaccessible nodes that are not preceded by an inaccessible node, d is 
the number of inaccessible nodes that are preceded by an inaccessible node, and w is 
the total number of words in the accessible nodes. If the memory contains n nodes, 
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with pn of them inaccessible, then we may estimate o = (1 — p)n, c = (1 — p)pn , 
d = p 2 n. Example: five-word nodes (on the average), with two link fields per node 
(on the average), and a memory of 1000 nodes. Then when p = |, it takes 374u per 
available node recovered; when p = |, it takes 104u; and when p = it takes only 33 u. 

36. A single customer will be able to sit in one of the sixteen seats 1, 3, 4, 6, ... , 23. 
If a pair enters, there must be room for them; otherwise there are at least two people 
in seats (1,2,3), at least two in (4,5,6), . . . , at least two in (19,20,21), and at least 
one in 22 or 23, so at least fifteen people are already seated. 

37. First sixteen single males enter, and she seats them. There are 17 gaps of empty 
seats between the occupied seats, counting one gap at each end, with a gap of length 
zero assumed between adjacent occupied seats. The total number of empty seats, 
namely the sum of all seventeen gaps, is 6. Suppose x of the gaps are of odd length; 
then 6 - x spaces are available to seat pairs. (Note that 6 - x is even and > 0.) Now 
each of the customers 1, 3, 5, 7, 9, 11, 13, 15, from left to right, who has an even gap 
on both sides, finishes his lunch and walks out. Each odd gap prevents at most one of 
these eight diners from leaving, hence at least 8 - x people leave. There still are only 
6 — x spaces available to seat pairs. But now (8 — x)/2 pairs enter. 

38. The arguments generalize readily; lV(n, 2) = [(3n - 1)/2J for n > 1. [When the 
hostess uses a first-fit strategy instead of an optimal one, Robson has proved that the 
necessary and sufficient number of seats is [(5 n — 2)/3j.] 

39. Divide memory into three independent regions of sizes N(n i,m), N(n 2l m), and 
N(2m - 2, m). To process a request for space, put each block into the first region for 
which the stated capacity is not exceeded, using the relevant optimum strategy for that 
region. This cannot fail, for if we were unable to fill a request for x locations we must 
have at least (ni — x + 1) + (n 2 — x + 1) + (2m — x — 1) > ni -\- n 2 — x locations already 
occupied. 

Now if f(n) = N(n, m) + N(2m — 2, m), we have the subadditive law /(m + 712 ) < 
f( n 1 ) + f{n 2 ). Hence lim/(n)/n exists. (Proof: f(a + be) < f(a) + bf(c); hence 
limsup^^^ f(n)/n = maxo< a < c limsup;,.^^ f (a, + be) / (a + be) < f(c)/c for all c; hence 
limsup n-yoa f(n)/n < lim inf,,-^ f(n)/n.) Therefore lim N(n,m)/n exists. 

[From exercise 38 we know that N( 2) = §. The value N(rn) is not known for any 
m > 2. It is not difficult to show that the multiplicative factor for just two block sizes, 

1 and 6, is 2 — 1/6; hence N(3) > l|. Robson’s methods imply that JV(3) < ly|, and 

2 < N(4) < 2|.[ 

40. Robson has proved that N(2 r ) < 1 + r, by using the following strategy: Allocate 
to each block of size k , where 2 m < k < 2 m+1 , the first available block of k locations 
starting at a multiple of 2 m . 

Let N({b\,b 2 , . . . ,b n }) denote the multiplicative factor when all block sizes are 
constrained to lie in the set { 61 , 62 , . . . , b n }, so that N(n) = 7V({1, 2, ... . , n}). Robson 

and S. Krogdahl have discovered that N({b 1 ,b 2 ,.. ,,b n }) — n- (bi/b 2 -\ hhn-i/M 

whenever bi is a multiple of bi - 1 for 1 < i < n; indeed, Robson has established the 
exact formula N(2 r m, { 1 , 2, 4, . . . , 2 r }) = 2 r m(l + |r) - 2 r + 1 . Thus in particular, 
N(n) > 1 + | [lg n\ . He also has derived the upper bound N(n) < 1.1825 Inn + 
0(1), and he conjectures tentatively that N(n) = H n . This conjecture would follow 
if N({b 1 , b 2 , ... , h n }) were equal to n — ( 61/62 + • • • + 6 n _i/ 6 n ) in general, but this is 
unfortunately not the case since Robson has proved that 1V({3,4}) >1^. (See Inf. 
Proc. Letters 2 (1973), 96-97; JACM 21 (1974), 491-499.) 
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41. Consider maintaining the blocks of size 2 k \ The requests for sizes 1, 2, 4, , 2 k ~ 1 

will periodically call for a new block of size 2 k to be split, or a block of that size will 
be returned. We can prove by induction on k that the total storage consumed by such 
split blocks never exceeds kn\ for after every request to split a block of size 2 k+1 , we are 
using at most kn locations in split 2 fc -blocks and at most n locations in unsplit ones. 

This argument can be strengthened to show that a r n cells suffice, where ao = 1 
and a/c = 1 + ajb-i(l — 2 ~ k ); we have 


k= 0 

a/c = 1 


1 2 3 4 5 

i 1 ol 9— o 697 a 18535 

2 8 ^ 64 °1024 ^ 32768 


Conversely for r < 5 it can be shown that a buddy system sometimes requires as many 
as a r n cells, if the mechanism of steps R1 and R2 is modified to choose the worst 
possible available 2 J -block to split instead of the first such block. 

Robson’s proof that N(2 r ) < 1 + r (see exercise 40) is easily modified to show 
that such a “leftmost” strategy will never need more than (1 + \r)n cells to allocate 
space for blocks of sizes 1,2,4, . . . , 2 r , since blocks of size 2 k will never be placed in 
locations > (1 + |fc)n. Although his algorithm seems very much like the buddy system, 
it turns out that no buddy system will be this good, even if we modify steps R1 and 
R2 to choose the best possible available 2 J -block to split. For example, consider the 


following sequence of “snapshots 

s” of the memory, for n 

= 16 and r = 

11111111 

11111111 

00000000 

00000000 

10101010 

10101010 

2-2-2-2- 

00000000 

11110000 

11110000 

2-110000 

00000000 

11111111 

11110000 

11110000 

00000000 

10101010 

10102-2- 

10102-2- 

00000000 

10001000 

10002-00 

10002-00 

4—4— 

10000000 

10000000 

10000000 

4—0000 


Here 0 denotes an available location and k denotes the beginning of a fc-block. In a 
similar way there is a sequence of operations, whenever n is a multiple of 16, that forces 
Y^n blocks of size 8 to be | full, and another to be | full. If n is a multiple of 
128, a subsequent request for n blocks of size 8 will require more than 2.5n memory 
cells. (The buddy system allows unwanted Is to creep into j^n of the 8-blocks, since 
there are no other available 2s to be split at a crucial time; the “leftmost” algorithm 
keeps all Is confined.) 

42. We can assume that m > 6. The main idea is to establish the occupancy pattern 
Rm- 2 {F m - 3 Ri) k at the beginning of the memory, for k = 0, 1, . . . , where Rj and Fj 
denote reserved and free blocks of size j. The transition from k to k + 1 begins with 


Rm — 2 ( F m — 3 Rl ) ^ — 1 Rm -2 {F m - 3 R 1 ) k R m _2 Rm- 2 

~ * Rm-2(Fm-3Rl) k 1 F2m-4Rm-2 
— > Rm — 2 (Fm — ^Rl ) k 1 RmRm-sRlRm-2 
—> Rm-2(Fm-3Rl) k 1 FmRm-sRi ; 


then the commutation sequence F m _ 3 RiF m R m _ 5 Ri — >• F m _ 3 RiR m _ 2 R 2 Rm- 5 Ri -> 
F 2 m--iR 2 Rm- 5 Ri — > RmRm-bR\R 2 Rm-bRi — f FmRm-sRiF m - 3 R\ is used k times 
until we get F rn R m _ 5 Ri(F rrl _ 3 i? 1 ) fc -> F 2 m-sRi{F m - 3 R 1 ) k ->■ i? m _ 2 (F m _ 3 Ri)' c+1 . 
Finally, when k gets large enough, there is an endgame that forces overflow unless the 
memory size is at least (n - 4m + ll)(m - 2); details appear in Comp. J. 20 (1977), 
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242-244. [Notice that the worst conceivable worst case, which begins with the pattern 
Fm-iRiFm-iRiFm-xRx ..., is only slightly worse than this; the next-fit strategy of 
exercise 6 can produce this pessimal pattern.] 

43. We will show that if Di, D 2 , . . . is any sequence of numbers such that Di/m. + 

D 2 /(m + 1) + 1- D m / (2m — 1) > 1 for all m > 1, and if C m = Di/1 + Do/2 + h 

Dm/m, then NpF(n,m) < nC m . In particular, since 

1 i 1 i i 1 i 1 1 1 1 

m m+ 1 + 2m + 1 ~ 2 H h 2m - 3 ~~ 2m - 2 + 2m. - 1 > ln 2 ’ 

the constant sequence D m = l/ln2 satisfies the necessary conditions. The proof is by 
induction on m. Let Nj = nCj for j > 1, and suppose that some request for a block 
of size m cannot be allocated in the leftmost N m cells of memory. Then m > 1 . For 
0 < j < m, we let N' denote the rightmost position allocated to blocks of sizes < j , 
or 0 if all reserved blocks are larger than j; by induction we have Nj < Nj. Furthermore 
we let be the rightmost occupied position < N m , so that N' m > N m - m + 1. 
Then the interval (N'_ 1 . . N- ] contains at least - N' j _ 1 )/(m + j - 1)] occupied 

cells, since its free blocks are of size < m and its reserved blocks are of size > j. It 
follows that n - m> number of occupied cells > X!"li j( N 'j - A^'_i)/(m + j~- 1) = 
mN' m /(2m - 1) - (m - 1) E”!'/ N//(m + j)(m + j - 1) > rn7V m /(2rn - 1) - m - 

( m - 1) E-r 1 Njil/im + j - 1) - 1/ (m + j)) = nDj/(m + j - i) - m > n - m, 
a contradiction. 

[This proof establishes slightly more than was asked. If we define the ZXs by 
Dx/m -4 b Dm/ (2m - 1) = 1, then the sequence Ci, C 2 , . . . is 1, |, ||||, . . . ; 

and the result can be improved further, even in the case m = 2, as in exercise 38.] 

44. [i^U/AOl, [F- 1 (2/fV)l, ..., \F~ 1 (N/N)~\. 
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TABLES OF NUMERICAL QUANTITIES 


Table 1 

QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES 
AND IN ANALYSIS OF COMPUTER PROGRAMS (40 DECIMAL PLACES) 



s/2 = 1.41421 

35623 

73095 

04880 

16887 

24209 

69807 

85697- 


s/3 = 1.73205 

08075 

68877 

29352 

74463 

41505 

87236 

69428+ 


s/E = 2.23606 

79774 

99789 

69640 

91736 

68731 

27623 

54406+ 


VlO = 3.16227 

76601 

68379 

33199 

88935 

44432 

71853 

37196- 


s/2 = 1.25992 

10498 

94873 

16476 

72106 

07278 

22835 

05703- 


v^3 = 1.44224 

95703 

07408 

38232 

16383 

10780 

10958 

83919- 


^2 = 1.18920 

71150 

02721 

06671 

74999 

70560 

47591 

52930- 


In 2 = 0.69314 

71805 

59945 

30941 

72321 

21458 

17656 

80755+ 


In 3 = 1.09861 

22886 

68109 

69139 

52452 

36922 

52570 

46475- 


In 10 = 2.30258 

50929 

94045 

68401 

79914 

54684 

36420 

76011+ 


1/ln 2 = 1.44269 

50408 

88963 

40735 

99246 

81001 

89213 

74266+ 


1/lnlO = 0.43429 

44819 

03251 

82765 

11289 

18916 

60508 

22944- 


7 r = 3.14159 

26535 

89793 

23846 

26433 

83279 

50288 

41972- 

1° 

= tt/180 = 0.01745 

32925 

19943 

29576 

92369 

07684 

88612 

71344+ 


1/tt = 0.31830 

98861 

83790 

67153 

77675 

26745 

02872 

40689+ 


7T 2 = 9.86960 

44010 

89358 

61883 

44909 

99876 

15113 

53137- 


= T(l/2) = 1.77245 

38509 

05516 

02729 

81674 

83341 

14518 

27975+ 


T(l/3) = 2.67893 

85347 

07747 

63365 

56929 

40974 

67764 

41287- 


r(2/3) = 1.35411 

79394 

26400 

41694 

52880 

28154 

51378 

55193+ 


e = 2.71828 

18284 

59045 

23536 

02874 

71352 

66249 

77572+ 


1/e = 0.36787 

94411 

71442 

32159 

55237 

70161 

46086 

74458+ 


e 2 = 7.38905 

60989 

30650 

22723 

04274 

60575 

00781 

31803+ 


7 = 0.57721 

56649 

01532 

86060 

65120 

90082 

40243 

10422- 


In 7r = 1.14472 

98858 

49400 

17414 

34273 

51353 

05871 

16473- 


4> = 1.61803 

39887 

49894 

84820 

45868 

34365 

63811 

77203+ 


e 7 = 1.78107 

24179 

90197 

98523 

65041 

03107 

17954 

91696+ 


e 7 ^ 4 = 2.19328 

00507 

38015 

45655 

97696 

59278 

73822 

34616+ 


sin 1 = 0.84147 

09848 

07896 

50665 

25023 

21630 

29899 

96226- 


cos 1 = 0.54030 

23058 

68139 

71740 

09366 

07442 

97660 

37323+ 


— C'(2) = 0.93754 

82543 

15843 

75370 

25740 

94567 

86497 

78979- 


C(3) = 1.20205 

69031 

59594 

28539 

97381 

61511 

44999 

07650- 


In </> = 0.48121 

18250 

59603 

44749 

77589 

13424 

36842 

31352- 


1/ln </> = 2.07808 

69212 

35027 

53760 

13226 

06117 

79576 

77422- 


— In In 2 = 0.36651 

29205 

81664 

32701 

24391 

58232 

66946 

94543- 
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Table 2 

QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES 
AND IN ANALYSIS OF COMPUTER PROGRAMS (45 OCTAL PLACES) 

The names at the left of the “=” signs are given in decimal notation. 


0.1 = 

0.06314 

0.01 = 

0.00507 

0.001 = 

0.00040 

0.0001 = 

0.00003 

0.00001 = 

0.00000 

0.000001 = 

0.00000 

0.0000001 = 

0.00000 

0.00000001 = 

0.00000 

0.000000001 = 

0.00000 

0.0000000001 = 

0.00000 

72 = 

1.32404 

73 = 

1.56663 

75 = 

2.17067 

TTo = 

3.12305 

72 = 

1.20505 

73 = 

1.34233 

72 = 

1.14067 

In 2 = 

0.54271 

In 3 = 

1.06237 

In 10 = 

2.23273 

1/ln 2 = 

1.34252 

1/lnlO = 

0.33626 

7T = 

3.11037 

1° = tt/180 = 

0.01073 

1 / 7T 

0.24276 

■ = 

11.67517 

Tif = r(i/2) = 

1.61337 

r(i/3) = 

2.53347 

r(2/3) = 

1.26523 

e = 

2.55760 

1/e = 

0.27426 

e 2 = 

7.30714 

7 = 

0.44742 

ln7r = 

1.11206 

<t> = 

1.47433 

e 7 = 

1.61772 

e 7 / 4 = 

2.14275 

sin 1 = 

0.65665 

cos 1 = 

0.42450 

II 

O? 

1 

0.74001 

C (3) = 

1.14735 

In <(> = 

0.36630 

1/ln <f) = 

2.04 776 

— In In 2 = 

0.27351 


63146 31463 14631 46314 
53412 1 7270 24365 60507 
61115 64570 65176 76355 
21556 13530 7 O414 54512 
24761 32610 70664 36041 
02061 57364 05536 66151 
00153 27745 15274 53644 
00012 57143 56106 04303 
00001 04560 27640 46655 
00000 06676 33766 35367 
74631 77167 46220 42627 
65641 30231 25163 54453 
36334 57722 4 7602 57471 
40726 64555 22444 02242 
05746 15345 05342 10756 
50444 2.2175 73134 67363 
7 4050 61556 12455 72152 
02775 75071 73632 57117 
24752 55006 05227 32440 
06735 52524 25405 56512 
16624 53405 77027 35750 
75425 11562 41614 52325 
55242 10264 30215 14230 
72152 11224 72344 25603 
30155 62344 20251 23760 
14467 62135 71322 25561 
61106 64736 6524 7 4 7035 
35234 51013 61316 73106 
57112 14154 74312 54572 
52130 50535 51246 52773 
53066 13167 46761 52726 
45615 23355 33460 63507 
14 770 67666 06172 23215 
4044 3 4 7503 36413 65374 
57156 27751 23701 27634 
13452 61152 65761 22477 
31512 16162 52370 35530 
24436 04414 73402 03067 
50037 32406 42711 07022 
45144 53253 42362 42107 
00023 6001 4 20470 15613 
26256 61213 01145 13700 
60111 17144 41512 11436 
71233 67265 63650 17401 


63146 31463 14631 46315- 
53412 1 7270 24365 60510- 
44264 16254 02030 44672+ 
75170 33021 15002 35223- 
06077 17401 56063 344 1 7- 
55323 07746 44470 26033+ 
12741 72312 20354 02151+ 
47374 77341 01512 63327+ 
12262 71426 40124 21742+ 
55653 37265 34642 01627 - 
66115 46725 12575 17435+ 
50265 60361 34073 42223- 
63003 00563 55620 32021- 
57101 41466 33775 22532+ 
65334 25574 22415 03024+ 
76133 05334 3114 7 60121- 
64430 60271 02755 73136+ 
07316 30007 71366 53640+ 
63065 25012 35574 55337+ 
66542 56026 46050 50705+ 
37766 40644 35175 04353+ 
33525 27655 14756 06220- 
63050 56006 70163 21122+ 
54276 63351 22056 11544+ 
47257 50765 15156 70067- 
15466 30021 4065 4 34103- 
40510 15273 34470 17762- 
47644 54653 00106 66O46- 
37655 60126 23231 02452+ 
42542 00471 72363 61661 + 
75436 02440 52371 03355+ 
35040 32664 25356 50217+ 
74376 01002 51313 25521- 
52661 52410 37511 46057+ 
71401 40271 66710 15010+ 
36553 53327 17554 21260+ 
11342 53525 44307 02171- 
23644 11612 07474 14505- 
14666 27320 70675 12321 + 
23350 50074 46100 27706+ 
42561 31715 10177 O66I4+ 
41004 52264 30700 4O646+ 
16575 00355 43630 40651+ 
56637 26334 31455 57005- 
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Several of the 40-digit values in Table 1 were computed on a desk calculator 
by John W. Wrench, Jr., for the first edition of this book. When computer 
software for such calculations became available during the 1970s, all of his 
contributions proved to be correct. See the answer to exercise 1.3.3-23 for the 
40-digit value of another fundamental constant. 


Table 3 


VALUES OF HARMONIC NUMBERS, BERNOULLI NUMBERS 
AND FIBONACCI NUMBERS, FOR SMALL VALUES OF n ’ 


n 

H n 

B n 

F n 

n 

0 

0 

1 

0 

0 

1 

1 

-1/2 

1 

1 

2 

3/2 

1/6 

1 

2 

3 

11/6 

0 

2 

3 

4 

25/12 

-1/30 

3 

4 

5 

137/60 

0 

5 

5 

6 

49/20 

1/42 

8 

6 

7 

363/140 

0 

13 

7 

8 

761/280 

-1/30 

21 

8 

9 

7129/2520 

0 

34 

9 

10 

7381/2520 

5/66 

55 

10 

11 

83711/27720 

0 

89 

11 

12 

86021/27720 

-691/2730 

144 

12 

13 

1145993/360360 

0 

233 

13 

14 

1171733/360360 

7/6 

377 

14 

15 

1195757/360360 

0 

610 

15 

16 

2436559/720720 

-3617/510 

987 

16 

17 

42142223/12252240 

0 

1597 

17 

18 

14274301/4084080 

43867/798 

2584 

18 

19 

275295799/77597520 

0 

4181 

19 

20 

55835135/15519504 

-174611/330 

6765 

20 

21 

18858053/5173168 

0 

10946 

21 

22 

19093197/5173168 

854513/138 

17711 

22 

23 

444316699/118982864 

0 

28657 

23 

24 

1347822955/356948592 

-236364091/2730 

46368 

24 

25 

34052522467/8923714800 

0 

75025 

25 

26 

34395742267/8923714800 

8553103/6 

121393 

26 

27 

312536252003/80313433200 

0 

196418 

27 

28 

315404588903/80313433200 

-23749461029/870 

317811 

28 

29 

92270465 11387/2329089562800 

0 

514229 

29 

30 

9304682830147/2329089562800 

8615841276005/14322 

832040 

30 
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For any x, let H x 



. Then 


# 1/2 — 2 — 2 In 2, 

#i /3 = 3 - |ir/\/3 - | In 3, 

#2/3 = I + - § In 3, 

# 1/4 = 4 — ^7r — 3 In 2, 

# 3/4 = | + — 3 In 2, 

# 1/5 = 5 - i7r^> 3/2 5~ 1/4 - | In 5 - |-\/51n0, 
# 2/5 = § - §7r<r 3/2 5- 1/4 - |ln5+ iV51n^, 
# 3/5 = | + | tt < 0 - 3 / 2 5-!/ 4 - f ln5+ Iv^ln^ 
# 4/5 = f + |7r(/> 3 / 2 5“ 1 / 4 - | ln5 - |\/51n</>, 
#i/6 ='6 — i7r v / 3-21n2- fin 3, 

# 5/6 = | + |7rV3-21n2- fin 3, 


and, in general, when 0 < p < q (see exercise 1.2.9-19), 



7T p 

— cot - 7T 
2 q 


In 2g + 2 


E 

l<n<q/2 


2pn . n 

cos 7r ■ lnsm — 7r. 

Q q 
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In the following formulas, letters that are not further qualified have the following 
significance: 

j, k integer- valued arithmetic expression 

to, n nonnegative integer-valued arithmetic expression 
x, y real-valued arithmetic expression 

/ real-valued or complex-valued function 

P pointer-valued expression (either A or a computer address) 

S , T set or multiset 

a string of symbols 


Formal symbolism 

Meaning 

Where 

defined 

V 4- E 

give variable V the value of expression E 

1.1 

U<^V 

interchange the values of variables U and V 

1.1 

A n or A[n\ 

the nth element of linear array A 

1.1 

A mn or A[m, n] 

the element in row m and column n of rect- 



angular array A 

1.1 

NODE(P) 

the node (group of variables that are indi- 
vidually distinguished by their field names) 



whose address is P, assuming that P / A 

2.1 

F (P) 

the variable in NODE(P) whose field name is F 

2.1 

CONTENTS (P) 

contents of computer word whose address is P 

2.1 

LOC(V) 

address of variable V within a computer 

2.1 

P 4= AVAIL 

set the value of pointer variable P to the 



address of a new node 

2.2.3 

AVAIL 4= P 

return NODE(P) to free storage; all its fields 



lose their identity 

2.2.3 

top(S) 

node at the top of a nonempty stack S 

2.2.1 

X 4= S 

pop up S to X: set X t— top(S); then delete 



top(S) from nonempty stack S 

2.2.1 

S 4= X 

push down X onto S: insert the value X as 



a new entry on top of stack S 

2.2.1 
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Formal symbolism 

Meaning 

Where 

defined 

{B=>E- E') 

conditional expression: denotes E if B is 
true, E' if B is false 


[B] 

characteristic function of condition B: 



(B => 1; 0) 

1.2.3 

$kj 

Kronecker delta: [j = k] 

1.2.3 

[*"]*(*) 

coefficient of z n in power series g(z) 

1.2.9 

E /( fc ) 

sum of all f(k) such that the variable k is an 


R{k) 

integer and relation R(k) is true 

1.2.3 

n /w 

product of all f(k) such that the variable k 


R(k) 

is an integer and relation R(k) is true 

1.2.3 

min/(fc) 

minimum value of all f(k) such that the var- 


R(k) 

iable k is an integer and relation R(k) is true 

1.2.3 

max/(fc) 

maximum value of all f(k) such that the var- 


R(k ) 

iable k is an integer and relation R(k) is true 

1.2.3 

j\k 

j divides k: k mod j = 0 and j > 0 

1.2.4 

S\T 

set difference: {a | a in S and a not in T} 


gcd (j, A;) 

greatest common divisor of j and k : 



( j — k = 0 ==> 0; max d ) 

\ d\j, d\k J 

1.1 

j ± k 

j is relatively prime to k : gcd(j, k) — 1 

1.2.4 

A t 

transpose of rectangular array A: 

A T [j,k] = A[k,j] 


a R 

left-right reversal of a 


x y 

x to the y power (when x is positive) 

1.2.2 

x k 

x to the fcth power: 

/ \ 



(k> 0 =t- x\ \/x~ k \ 

' 0 <j<fc / 

1.2.2 

x k 

x to the k rising: T(a: + k)/T(x) — 

/ \ 



(k>0 PJ (x + j); l/(x + k)~ k J 

V 0 <j<k / 

1.2.5 

x k - 

x to the k falling: x\/(x — k)\ = 

/ \ 



(k>0=> (x-j); 1/ (x — k)— j 

' 0 <j<k / 

1.2.5 
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Formal symbolism 

Meaning 

Where 

defined 

n\ 

n factorial: T(n + 1) = n- 

1.2.5 

C) 

binomial coefficient: (k < 0 => 0; x-/k\) 

1.2.6 

( n ) 

\ni, ^2 > • • • > / 

multinomial coefficient (defined only when 
n = ni + n 2 + ■ ■ ■ + n m ) 

1.2.6 

* 71 1 
_mj 

Stirling number of the first kind: 



^ v k\k2 • • • fc n _ m 

0<fci <k2<--‘<k n - Tn <n 

1.2.6 

{”} 

Stirling number of the second kind: 



^ ^ ^1^2 •• • kn — m 

1</Cl < Al2 < ■ • • </c n _ m <771 

1.2.6 

{« 1 R{a)} 

set of all a such that the relation JJ(a) is true 


{ a l , ■ • • , c i n ]■ 

the set or multiset { a 1 < k < n} 


M 

fractional part (used in contexts where a 
real value, not a set, is implied): x — [xj 

1.2.11.2 

a l + 02 + ' ' ' + dn 

n-fold sum: a j 

1.2.3 

[a..b] 

closed interval: {x | a < x < 6} 

1.2.2 

( a-b ) 

open interval: {x | a < x < 6} 

1.2.2 

[a- -b) 

half-open interval: {x \ a < x < b} 

1.2.2 

(a..b\ 

half-closed interval: {x \ a < x < b} 

1.2.2 

\s\ 

cardinality: the number of elements in set S 


M 

absolute value of x: (x > 0 => x; -x) 


M 

length of a 


W 

floor of x, greatest integer function: maxfc< x fc 

1.2.4 

M 

ceiling of x, least integer function: minfc> x k 

1.2.4 

x mod y 

mod function: (j) = 0 ==> x; x — y [x/y \ ) 

1.2.4 

x = x 1 (modulo y) 

relation of congruence: x mod y = x' mod y 

1.2.4 

0(f(n)) 

big-oh of f(n), as the variable n — > oo 

1.2.11.1 

0(f(z )) 

big-oh of f(z), as the variable 2 — »• 0 

1.2.11.1 

n(f(n)) 

big-omega of /(n), as the variable n — >■ 00 

1.2.11.1 

e(/(n)) 

big-theta of f(n), as the variable n — > 00 

1.2.11.1 
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Formal symbolism 

Meaning 

Where 

defined 

l°gb x 

logarithm, base b , of x (when a; > 0, 



b > 0, and 6 / 1): the y such that x = b v 

1.2.2 

In a; 

natural logarithm: log e x 

1.2.2 

lg x 

binary logarithm: log 2 x 

1.2.2 

expx 

exponential of x: e x 

1.2.9 

(Xn) 

the infinite sequence Xq, Xi, X 2 , ... 



(here the letter n is part of the symbolism) 

1.2.9 

fix) 

derivative of / at x 

1.2.9 

f"(x) 

second derivative of / at x 

1.2.10 

f {n \x) 

nth derivative: (n — 0 => f(x)\ g'(x)), 



where g(x ) = /( n-1 )( x) 

1.2.11.2 

H { n x) 

harmonic number of order x: l/k x 

1 <fc<n 

1.2.7 

H n 

harmonic number: 

1.2.7 

Fn 

Fibonacci number: 



(n < 1 => n; F n _ x + F n _ 2 ) 

1.2.8 

B n 

Bernoulli number: n! [z n ] z/(e z — 1) 

1.2.11.2 

det(A) 

determinant of square matrix A 

1.2.3 

sign(x) 

sign of x: [x > 0] — [x < 0] 


C( x ) 

zeta function: lim^oo Hn X) (when x > 1) 

1.2.7 

r(x) 

gamma function: (x - 1)! = 7(2,00) 

1.2.5 

7 (x,y) 

incomplete gamma function: e~ t t x ~ 1 dt 

1.2.11.3 

7 

Euler’s constant: lim n _ +O0 (if n — Inn) 

1.2.7 

e 

base of natural logarithms: Y2 n >o V n - 

1 .2.2 

7 r 

circle ratio: 4 J2 n>0 (-l) n /(2n + i) 

1 .2.2 

oo 

infinity: larger than any number 


A 

null link (pointer to no address) 

2.1 

e 

empty string (string of length zero) 


0 

empty set (set with no elements) 


<A 

golden ratio: |(l + \/5) 

1 .2.8 

V>(n) 

Euler’s totient function: ^ [fcj_n] 

0<fc<n 

1.2.4 

x « y 

x is approximately equal to y 

1.2.5, 4.2.2 
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Formal symbolism 

Meaning 

Where 

defined 

Pr(S(X)) 

probability that statement S(X) is true, for 



random values of X 

1 . 2.10 

EX 

expected value of X: x Pr(X = x) 

1 . 2.10 

mean (g) 

mean value of the probability distribution 



represented by generating function g : g'( 1 ) 

1 . 2.10 

var(gf) 

variance of the probability distribution 
represented by generating function g: 



</'(!) + </(!)-</( I ) 2 

1 . 2.10 

(min xi, ave x 2 , 

a random variable having minimum 


max X 3 , dev X4) 

value xi, average (expected) value x 2 , 



maximum value X 3 , standard deviation X\ 

1 . 2.10 

P* 

address of preorder successor of NODE(P) in 



a binary tree or tree 

2.3.1, 2.3.2 

P$ 

address of inorder successor of NODE(P) in a 



binary tree, postorder successor in a tree 

2.3.1, 2.3.2 

Ptt 

address of postorder successor of NODE(P) in 



a binary tree 

2.3.1 

*P 

address of preorder predecessor of NODE(P) 



in a binary tree or tree 

2.3.1, 2.3.2 

$P 

address of inorder predecessor of NODE(P) in 



a binary tree, postorder predecessor in a tree 

2.3.1, 2.3.2 

IIP 

address of postorder predecessor of NQDE(P) 



in a binary tree 

2.3.1 

1 

end of algorithm, program, or proof 

1.1 

U 

one blank space 

1.3.1 

rA 

register A (accumulator) of MIX 

1.3.1 

rX 

register X (extension) of MIX 

1.3.1 

rll, . . . , rI 6 

(index) registers 11, . . . , 16 of MIX 

1.3.1 

rJ 

(jump) register J of MIX 

1.3.1 

(L:R) 

partial field of MIX word, 0 < L < R < 5 

1.3.1 

OP ADDRESS, 1(F) 

notation for MIX instruction 

1.3.1, 1.3.2 

u 

unit of time in MIX 

1.3.1 

* 

“self” in MIXAL 

1.3.2 

OF, IF, 2F, . . . , 9F 

“forward” local symbol in MIXAL 

1.3.2 

OB, IB, 2B, ..., 9B 

“backward” local symbol in MIXAL 

1.3.2 

OH, 1H, 2H, . . . , 9H 

“here” local symbol in MIXAL 

1.3.2 
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Some Men pretend to understand a Book 
by scouting thro’ the Index: 
as if a Traveller should go about to describe a Palace 
when he had seen nothing but the Privy. 

— JONATHAN SWIFT, Mechanical Operation of the Spirit (1704) 

When an index entry refers to a page containing a relevant exercise, see also the answer to 
that exercise for further information. An answer page is not indexed here unless it refers to a 
topic not included in the statement of the exercise. 


(), 164, see Identity permutation. 

0-2-trees, 317. 
oriented, 398. 

0-origin indexing, 254, 282, 299-301. 

305-306. 

2-d trees, 564. 

7 (Euler’s constant), 75, 107, 114, 619-620. 
7r (circle ratio), 21, 619-620. 
as “random” example, 397. 

Wallis’s product for, 52, 116. 

<j> (golden ratio), 13, 18, 21, 80, 83-86, 
619-620. 

A-register of MIX, 125. 

Aardenne-Ehrenfest, Tatyana van, 375, 379. 
Aarons, Roger M., 528. 

Abel, Niels Henrik, 58, 498. 

binomial theorem, 58, 71-73, 398. 
limit theorem, 95. 

Absolute error, 116. 

Absolute value, 21. 

Absolutely convergent series, 29. 

ACE computer, 193, 229. 

Pilot, 230. 

Acerbi, Fabio, 593. 

Adams, Charles William, 230. 

ADD, 131-132, 208. 

Add to list: see Insertion. 

Addition of polynomials, 275-280, 357-359. 
Address: A number used to identify a 
position in memory, 
field of MIXAL line, 145, 151-153. 
of node, 233. 

portion of MIX instruction, 127. 

Address transfer operators of MIX, 

133-134, 210. 

Adjacent vertices of a graph, 363. 

Adobe Systems, 202. 

Agenda, 288, 291, 296, see Priority queue. 
Aho, Alfred Vaino, 560. 

Ahrens, Wilhelm Ernst Martin Georg, 162. 
Alhazen, see Ibn al-Haytham. 
al-Khwarizmi, Abu ‘Abd Allah 
Muhammad ibn Musa 
(^-oJjIyAjI Sll l ^jl), 

1, 79. 


ALF (alphabetic data), 151, 152, 155. 
Algebraic formulas, 313. 

differentiation, 90, 338-347, 459. 
manipulation of, 459-462. 
representation as trees, 337, 459. 
simplification of, 339, 347. 

ALGOL language, viii, 202, 229. 
Algorithm, origin of word, 1-2. 
Algorithms, 1-9. 

analysis of, vi, 7, 96-107, 170-172, 179, 
250, 253, 268, 278-280, 324-326, 331. 
380-382, 444-445, 451. 
communication of, 16. 
effective, 6, 8, 9. 
equivalence between, 467. 
form of in this book, 2-4. 
hardware-oriented, 26, 252, 611. 
how to read, 4, 16. 

proof of, 5-6, 13-17, 321, 361, 422, 434. 
properties of, 4-6, 9. 
random paths in, 380-381. 
set-theoretic definition, 7-9. 
theory of, 7, 9. 

Allocation of tables, see Dynamic storage 
allocation, Linked allocation. 
Representation, Sequential allocation. 
Alpern, Steven Robert, 526. 

Alphameric character: A letter, digit, or 
special character symbol, 
codes for MIX, 136, 138, 140-141. 

AMM: American Mathematical Monthly , 
published by the Mathematical 
Association of America since 1894. 
Amortized running time, 254. 

Analysis of algorithms, vi, 7, 96-107, 

170-172, 179, 250, 253, 268, 278-280, 
324-326, 331, 380-382, 444-445, 451. 
Analytical Engine, 1, 229. 

Ancestor, in a tree structure, 311, 348. 
Andre, Antoine Desire, 536. 

Anticipated input, 216, see Buffering. 
Antisymmetric relation, 261. 

Apostol, Tom Mike, 28. 

Arabic mathematics, 1, 162. 

Araujo, Saulo, 565. 
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Arbogast, Louis Frangois Antoine, 52, 105. 

Arborescences, 363, see Oriented trees. 

Arc digraph, 379. 

Arc in a directed graph, 372. 

Area of memory, 435. 

Arguments of subroutines, 187, 189. 

Arithmetic: Addition, subtraction, 

multiplication, and division, vi, ix. 
fixed point, 158. 
floating point, 131, 306. 
operators of MIX, 131-133, 208. 
polynomial, 275-280, 357-359, 361. 
scaled decimal, 160-161. 

Arithmetic expressions, see Algebraic 
formulas. 

Arithmetic progression, sum of, 11, 

31 32, 56. 

Array: A table that usually has a 

fc-dimensional rectangular structure, 

4, 232, 298-307. 
linked, 301-307. 

one-dimensional, see Linear list, 
represented as a tree, 315. 
sequential allocation, 159, 299-301, 
305-307. 

tetrahedral, 300-301, 306. 
two-dimensional, see Matrix, 
uninitialized, 307. 

Arrows, used to represent links in 
diagrams, 234. 

Assembly language: A language that is 
intended to facilitate the construction 
of programs in machine language by 
making use of symbolic and mnemonic 
conventions to denote machine 
language instructions, 
contrasted with high-level language, 

236, 535. 
for MIX, 144-157. 

Assembly program, 145, 153. 

Assertions, inductive, 15-20. 

Assigning a buffer, 219-223, 226-227. 

Assignment operation (<— ), 3. 

Asterisk (“*”) in assembly language, 

146, 149, 153. 

Asymmetric relations, 261. 

Asymptotic values: Functions that express 
the limiting behavior approached by 
numerical quantities, 
derivation of, 107-123, 243, 396-397, 

525, 565. 

Atom (in a List), 315, 408-412, 418. 
purpose of, 410. 

Automata theory, 230, 240, 463-464. 

Automaton: An abstract machine that is 
defined formally, often intended to 
be a model of some aspects of actual 
computers (plural: Automata), 463-464. 

AVAIL stack: Available space list, 256. 


Available space list, 256-261, 266, 269, 

278, 291, 413-414, 435-444. 
history, 458. 

variable-size blocks, 435-456. 

Average value of a random variable, 

97-98, 103. 

from a generating function, 100-103. 

Babbage, Charles, 1, 229. 

Bachmann, Paul Gustav Heinrich, 107. 
Backus, John Warner, 230. 

Bailey, Michael John, 462. 

Baker, Henry Givens, Jr., 605. 

Balanced directed graph, 374-375. 

Ball, Thomas Jaudon, 369. 

Ball, Walter William Rouse, 162. 

Ballot problem, 536-537. 

Barnes, Ernest William, 507. 

Barnett, Michael Peter, 462. 

Barrington (= Mix Barrington), David 
Arno, 526. 

Barry, David McAlister, subtle reference 
to, xi, 274. 

Barton, David Elliott, 66, 536. 

Base address, 244. 

Bead, 233, see Node. 

Before and after diagrams, 260-261, 

278, 281, 571. 

Beigel, Richard, 467. 

Bell, Eric Temple, 87. 

Bell Interpretive System, 230. 

Bellman, Richard Ernest, xv. 

Bendix G20, 124. 

Bennett, John Makepeace, 230. 

Berger, Robert, 385. 

Bergeron, Frangois, 395. 

Bergman, George Mark, 495, 593. 

Berlekamp, Elwyn Ralph, 273. 

Berman, Martin Fredric, 523. 

Berners-Lee, Mary Lee nee Woods, 230. 
Bernoulli, Daniel, 83. 

Bernoulli, Jacques (= Jakob = James), 

112, 115. 

numbers B n , 76, 91, 112-115. 
numbers, table, 621. 
polynomials, 44, 113-115, 503. 

Bernoulli, Jean (= Johann = John), III, 495. 
Bertrand, Joseph Louis Frangois, 
postulate, 512. 

Berztiss, Alfs Teodors, 462. 

Best-fit method of storage allocation, 
436-437, 447, 453-456. 

Beta function_B(x, y), 72. 

Bhaskara II, Acarya, son of Mahesvara 
0Hlf<M.N!4, Rlt^np-), 54. 

Bienayme, Irenee Jules, 98. 

Bienstock, Daniel, 514. 

Big-Oh notation, 107-111, 118. 

Big-Omega notation, 110-111. 

Big-Theta notation, 110. 

Bigelow, Richard Henry, 562. 
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Binary computer: A computer that 

manipulates numbers primarily in the 
binary (radix 2) number system. 

Binary logarithms, 23, 26. 

Binary number system, 24-26. 

Binary operator: A function of two 
variables, 337. 

Binary trees, 311-312, 317, 318-337, 

363, 459. 

complete, 401, 405, 563. 
copying of, 329-330, 333, 347. 
correspondence to trees and forests, 
334-335, 346. 
definition of, 312. 

“Dewey” notation for, 317, 331, 346, 405. 
enumeration of, 388-389, 405. 
equivalent, 328-329. 
erasing of, 333. 
extended, 399—406. 
linked, 318. 
oriented, 396-397. 
path length of, 399-406. 
representation of, 318, 322, 327, 333-334. 
right-threaded, 327, 332-334. 
sequential, 401. 
similar, 327-329. 
threaded, 322, 331-332, 460. 
traversal of, 319-320, 323, 331-332, 
459-460. 

with shared subtrees, 326, 603. 

Binet, Jacques Philippe Marie, 36, 

407, 475, 582. 

Binomial coefficients, 52-74, 89. 
asymptotic values, 72. 
bounds on, 74. 

combinatorial interpretation, 52-53, 73. 
defined, 53. 

generalized, 65, 72, 85. 
generating functions, 90, 94. 
history, 53-54. 
inequalities involving, 74. 
sums involving, 56-74, 76-78, 85, 96. 
table, 54. 

Binomial distribution, 101-102. 
tail, 106. 

Binomial number system, see Combinatorial 
number system. 

Binomial theorem, 57-58, 90. 

Abel’s generalization, 58, 71-73, 398. 
generalizations of, 64, 70-73, 90, 

398-399, 488. 

Hurwitz’s generalization, 399, 488. 
Binomial tree, xx. 

Bipartite trees, 398. 

Bit: “Binary digit”, either zero or unity. 

BIT: Nordisk Tidskrift for Informations- 
Behandling, an international journal 
published in Scandinavia since 1961. 
Bitwise operations, 442, 455, 510, 553. 
Blaauw, Gerrit Anne, 458. 


Blikle, Andrzej Jacek, 329. 

Block of external data, 136-137. 

Block of memory, 435. 

Blocking of records, 218, 225. 

Bobrow, Daniel Gureasko, 421, 460, 461. 
Boles, David Alan, 452. 

Bolzano, Bernard Placidus Johann 
Nepomuk, 382. 

Boncompagni, Prince Baldassarre, 79. 
Boolean functions, 526-527. 

Boothroyd, John, 177. 

Bootstrapping, 143. 

Borchardt, Carl Wilhelm, 406, 583. 

Bottom of stack, 241. 

Bottom-up process, 309, 351. 

Boundary tag method of storage allocation, 
440-442, 453-454, 461. 

Bracket notation for coefficients, 92. 

Bracket notation for logical statements, 

32 33, see Iverson’s convention. 

Branch instruction: A conditional 
“jump” instruction. 

Branch node of tree, 308. 

Breadth-first search, 351. 

Brenner, Norman Mitchell, 523. 

Brent, Richard Peirce, 563. 

Briggs, Henry, 26. 

Broline, Duane Marvin, 601. 

Brother, in a tree structure, 311. 

Brouwer, Luitzen Egbertus Jan, 406. 

Bruijn, Nicolaas Govert de, 121, 122, 375, 
379, 380, 478, 504, 543, 565. 

Buddy system for storage allocation, 
442-444, 447-448, 454-456, 461. 
Buffering of input-output, 158, 216-228. 
history, 231. 

swapping, 147, 159, 217-218, 225. 

Bugs: Errors or defects; see Debugging. 
Burke, John, 310. 

Burks, Arthur Walter, 359. 

Burleson, Peter Barrus, 462. 

Burroughs B220, 124. 

Burroughs B5000, 461. 

Busche, Conrad Heinrich Edmund 
Friedrich, 43. 

Busy waiting, 216. 

Byte: Basic unit of data, usually associated 
with alphameric characters, 125. 
in MIX, 124-125, 139. 

Byte size in MIX: The number of distinct 
values that might be stored in a byte. 

C language, 233, 556. 

Cache, 528. 

CACM: Communications of the ACM , 
a publication of the Association for 
Computing Machinery since 1958. 
Cajori, Florian, 24. 

Calendar, 160. 

California Institute of Technology, x, 282. 
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Call: To activate another routine in 
a program. 

Calling sequence, 187-190, 193, 196-197. 

Campbell, John Arthur, 450. 

Canonical cycle notation for permutations, 
178-179. 

Canonical representation of oriented trees, 
390-394, 397-398, 590-591. 

Capelli, Alfredo, 50. 

Car: LISP terminology for the first 
component of a List; analogous to 
INFO and DLINK on page 411, or to 
ALINK on page 415. 

Cards, playing, 51, 69, 233-237, 238, 
377-378. 

Cards, punched, 136-137, 152, 229. 

Carlitz, Leonard, 96, 499, 506. 

Carlyle, Thomas, xiv. 

Carpenter, Brian Edward, 229. 

Carr, John Weber, III, 458. 

Case Institute of Technology, i. 

Cassini, Gian (= Giovanni = Jean) 
Domenico (= Dominique), 81. 

Catalan, Eugene Charles, 407. 

numbers, 407. 

Cate, Esko George, 523. 

Cauchy, Augustin Louis, 92, 475, 490, 

506, 520, 582. 

inequality: (X>*A ) 2 < (E a fc)(E b k)> 
36. 

matrix, 37-38, 475. 

Cayley, Arthur, 396, 406-407, 586, 597. 

CDC 1604 computer, 124, 529. 

Cdr: LISP terminology for the remainder of 
a List with its first component deleted; 
analogous to RLINK on page 411, or 
to BLINK on page 415. 

Ceiling function [x], 39, 41. 

Cell: A word of the computer memory, 127. 

Cellar, 240. 

Central moment of a probability 
distribution, 105. 

Centroid of a free tree, 387-388, 397, 589. 

Chain: A word used by some authors to 
denote a linked linear list, by others 
to denote a linearly ordered set. 

Chain rule for differentiation, 52. 

Chaining: A word used by some authors 
in place of “linking”. 

Chakravarti, Gurugovinda ('3<K5ilfa«tT 
Wl-stST), 53. 

Change of summation variable, 28, 32-33. 

Channel: A data-transmission device 
connected to a computer, 224. 

CHAR (convert to characters), 138. 

Character codes of MIX, 136, 138, 140-141. 

Characteristic function of a probability 
distribution, 103. 

Characteristic polynomial of a matrix, 499. 


Charles Philip Arthur George of Edinburgh, 
Prince of Wales, 310. 

Cheam, Tat Ong, 450. 

Cheating, 582. 

Chebyshev, Pafnutii Lvovich (He6biineB, 
IlacJmyTHH JlbBOBnu), inequality, 

98, 104. 

Checkerboard, 435-436. 

Checkerboarding, see Fragmentation. 

Chen, Tien Chi (H^{g), 471. 

Cheney, Christopher John, 421. 

Cheney, Ednah Dow Littlehale, 377. 

Chernoff, Herman, 502. 

Chess, 6, 194, 272. 

Chia Hsien (M^), 53. 

Child link, 427-433. 

Children in tree structures, 311, 317, 
334-335, 352. 

Chinese mathematics, 53, 59, 75, 407. 

Chowla, Paromita (^tta&ol CTS'TT or 
mOTHdl ’"I NH I ), 307. 

Christian IX, King of Denmark, 310, 

311, 562. 

Christie Mallowan, Agatha Mary Clarissa 
Miller, xvii. 

Chu Shih-Chieh (= Zhu Shijie, Zhu 

Songtfng; 53, 59, 70. 

Chung Graham, Fan Rong King 
(»££#), 514. 

Chung, Kai Lai (11^^), 105. 

Cl: The comparison indicator of MIX, 126, 
134, 142, 211, 213, 228. 

Circle of buffers, 218-227, 231. 

Circular definition, 263, 308, see 
Definition, circular. 

Circular linking, 273-280, 302, 355, 411, 459. 

Circular store, 240. 

Circulating shift, 135. 

CITRUS subroutine library, 457. 

Clark, Douglas Wells, 604. 

Clavius, Christopher, 159. 

Clock, for real time, 228. 

Clock, simulated, 283, 288. 

Clock, solitaire game, 377-378. 

Closed subroutine, see Subroutine. 

CMath: Concrete Mathematics, a book 
by R. L. Graham, D. E. Knuth, and 
O. Patashnik, 11. 

CMP1 (compare rll), 134, 210-211. 

CMP A (compare rA), 134, 210-211. 

CMPX (compare rX), 134, 210-211. 

COBOL language, 424-434, 457, 458. 

Codes for difficulty of exercises, xvii. 

Coding: Synonym for “programming”, but 
with even less prestige. 

Coefficient extraction, 92. 

Cofactor of element in square matrix: 
Determinant of the matrix obtained 
by replacing this element by unity and 
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replacing all other elements in the same 
row or column by zero, 37, 381. 

Coffman, Edward Grady, Jr., 450-451. 

Cohen, Jacques, 421, 461, 614. 

Coin tossing, 101-102. 
tail of distribution, 106. 

Collins, George Edwin, 461. 

Combinations of n objects taken k at a 
time, 52-53, 69, 94. 
with repetitions permitted, 73, 92, 

95, 386, 388. 

with restricted repetitions, 95. 

Combinatorial matrix, 37-38, 589. 

Combinatorial number system, 73, 560. 

Comfort, Webb T., 461. 

COMIT language, 461. 

Command: Synonym for “instruction”. 

Comments, 2-3. 

in assembly language, 145, 149. 

Commutative law, 165. 

Comp. J.: The Computer Journal, a 

publication of the British Computer 
Society since 1958. 

Compacting memory, 423, 439, 449, 

452, 455. 

Comparability, 270. 

Comparison indicator of MIX, 126, 134, 

142, 211, 213, 228. 

Comparison operators of MIX, 134, 210-211. 

Compiler: A program that translates 
computer languages, 
algorithms especially for use in, 360, 
424-434, 556. 

Complete binary tree, 401, 405, 563. 

Complete t-ary tree, 401-402. 

Complex conjugate, 21. 

Complex number, 21. 

Composition of permutations, see 
Multiplication of permutations. 

Compound interest, 23-24. 

Compression of messages, 407. 

Computational error, 24-26. 

Computational method, 5, 7-8. 

Compute: To process data. 

Computer: A data processor. 

Computer language, see Assembly language, 
Machine language, Programming 
language. 

COM (constant), 149-150, 155. 

Concatenation of strings, 274. 

Concave function, 406. 

Conditional expression, 460, 624. 

Congruence, 40-42. 

Connected directed graph, 372. 
strongly, 372, 377. 

Connected graph, 363. 

Conservative law, 170, see Kirchhoff’s law. 

Constants in assembly language, 

149-150, 155. 


Construction of trees, 340-341, 343, 

428-429. 

CONTENTS, 127, 235-236. 

Context-free grammar, 539. 

Continuants, 600-601. 

Continued fractions, 498, 565. 

Continuous simulation, 282, 298. 

Convergence: An infinite sequence (X n ) 
converges if it approaches a limit as 
n approaches infinity; an infinite sum 
or product is said to “converge” or 
to “exist” if it has a value according 
to the conventions of mathematical 
calculus; see Eq. 1.2.3-(3). 
absolute, 29. 
of power series, 87, 396. 

Conversion operations of MIX, 138. 

Convex function, 404, 596. 

Convolution of probability distributions: 

The distribution obtained by adding 
two independent variables, 103. 

Conway, Melvin Edward, 151. 229. 

Conway, John Horton, 19, 80, 273, 408, 600. 

Copy a data structure: To duplicate a 

structured object by producing another 
distinct object that has the same data 
values and structural relationships, 
binary tree, 329-330, 333, 347. 
linear list, 279. 

List, 423. 

two-dimensional linked list, 306. 

Copying and compacting, 421. 

Corless, Robert Malcolm, 395. 

Coroutines, 193-200, 222-223, 283-296, 320. 
history, 229. 

linkage, 194, 200, 223, 291. 

Correspondence between binary trees and 
forests, 334-335, 346. 

Cousins, 317. 

Coxeter, Harold Scott Macdonald. 80, 

162, 407, 408. 

Crain, Steven Paul, 613. 

Crelle: Journal fiir die reine und angewandte 
Mathematik , an international journal 
founded by A. L. Crelle in 1826. 

Crelle, August Leopold, 58, 632. 

Critical path time, 217. 

Crossword puzzle, 163. 

Crowe, Donald Warren, 601. 

Cullen, Christopher, 75. 

Cumulants of a probability distribution, 
103-106. 

Cycle: Path from a vertex to itself, 
detection of, 271. 
fundamental, 366-370, 377. 

Hamiltonian, 374, 379. 
in directed graph, 363. 
in graph, 363. 

in permutation, 164-167, 176-178, 

182-184. 
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in random permutation, 179-184. 
notation for permutations, 164-167, 
172-175, 182. 

oriented, in directed graph, 372. 
singleton, in permutation, 164, 171, 
180-181. 

Cycle lemma, 593. 

Cyclic shift, 185. 

Dahl, Ole-Johan, 229, 230, 461, 462. 

Dahm, David Michael, 433, 434. 

Data (originally the plural of “datum”, 
but now used collectively as singular 
or plural, like “information”): 
Representation in a precise, formalized 
language of some facts or concepts, 
often numeric or alphabetic values, 
to facilitate manipulation by a 
computational method, 215. 
packed, 128, 158. 

Data organization: A way to represent 
information in a data structure, 
together with algorithms that access 
and/or modify the structure. 

Data structure: A table of data including 
structural relationships, 232-465. 
linear list structures, 238-298. 

List structures, 408-423. 
multilinked structures, 424-434. 
orthogonal lists, 298-307, 424-434. 
tree structures, 308-408. 

Daughter, in a tree structure, 311. 

David, Florence Nightingale, 66. 

Davies, David Julian Meredith, 445. 

Davis, Philip Jacob, 50. 

Dawson, Reed, 584. 

de Bruijn, Nicolaas Govert, 121, 122, 375, 
379, 380, 478, 504, 543, 565. 

de Moivre, Abraham, 74, 83, 87, 106, 

182, 474, 536. 

De Morgan, Augustus, 17. 

Deallocation, see Liberation. 

Debugging: Detecting and removing 
bugs (errors), 192-193, 201, 257, 

297, 413, 556. 

DEC1 (decrease rll), 134, 210. 

DECA (decrease rA), 134, 210. 

Decimal computer: A computer that 

manipulates numbers primarily in the 
decimal (radix ten) number system. 

Decimal number system, 21, 619. 

DECX (decrease rX), 134, 210. 

Defined symbol, an assembly language, 153. 

Definition, circular, see Circular definition. 

Degree, of node in tree, 308, 317, 377. 
of vertex in directed graph, 372. 

Deletion of a node: Removing it from a 
data structure and possibly returning 
it to available storage, 
from available space list, see Reservation. 


from deque, 251, 297. 

from doubly linked list, 281, 290-291, 297. 

from doubly linked ring structure, 358. 

from linear list, 239. 

from linked list, 236, 255, 276, 305. 

from queue, 242, 244-245, 254, 261, 

265, 273-274. 

from stack, 241, 242, 244-245, 247, 254, 
259, 269, 273-274, 278, 458. 
from tree, 358. 

from two-dimensional list, 305. 

Demuth, Howard B., 120. 

Depth-first search, 578, 581. 

Deque: Double-ended queue, 239-243, 269. 
deletion from, 251, 297. 
input-restricted, 239-243, 416. 
insertion into, 251, 297. 
linked allocation, 280, 297. 
output-restricted, 239-243, 269, 274. 
sequential allocation, 251. 

Derangements, 180, 183. 

Derivative, 90, 338. 

Dershowitz, Nachum (Viwn Diro), 

518, 588, 598. 

Descendant, in a tree structure, 311, 348. 
Determinant of a square matrix, 37-39, 

81, 378-379, 382. 

Deuel, Phillip DeVere, Jr., 556. 

Deutsch, Laurence Peter, 418, 421, 422. 
Dewar, Robert Berriedale Keith, 614. 

Dewey, Melvil, notation for binary trees (due 
to Galton), 317, 331, 346, 405. 
notation for trees, 313, 317, 382-383, 460. 
Diaconis, Persi Warren, 491. 

Diagonals of polygons, 408. 

Diagrams of structural information, 

234, 279. 

before-and-after, 260-261, 278, 281, 571. 
binary trees, 312, 318, 563. 

List structures, 315- 317, 408-409. 
tree structures, 309-315, 337, 346, 

349, 460. 

Dickman, Karl Daniel, 525. 

Dickson, Leonard Eugene, 81, 484. 
Dictionaries of English, 1-2, 215-216. 
Differences of polynomials, 64. 
Differentiation, 90, 338-347, 459. 
chain rule for, 52. 

Digamma function 'ip(z), 44, 75, 493. 

Digit: One of the symbols used in radix 
notation; usually a decimal digit, one 
of the symbols 0, 1, . . . , or 9. 

Digraphs, 372, see Directed graphs. 

Dijkstra, Edsger Wybe, 17, 191, 230, 231, 
240, 459, 462, 545, 580, 605. 
d’Imperio, Mary Evelyn, 462. 

Directed graphs, 372-374, 422. 
as flow charts, 364-365, 377. 
balanced, 374-375. 
connected, 363. 
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regular, 379. 

strongly connected, 372, 377. 

Discrete system simulation, 203, 282-298. 

synchronous, 282, 298. 

Disjoint sets: Sets with no common 
elements. 

Disk files, 136-137, 435, 463. 

Disk input, buffered, 228. 

Disposal, see Garbage collection, Liberation. 
Dissection of a polygon, 408. 

Distributed-fit method of storage allocation, 
450-451, 456. 

Distribution: A specification of probabilities 
that govern the value of a random 
variable. 

binomial, 101-102. 
negative binomial, 107. 
normal, 104-106, 122. 

Poisson, 106, 502, 524. 
tails of, 104, 106-107. 
uniform, 102, 253, 446. 

Distributive law, 28, 37, 42, 598. 

DIV (divide), 131-133, 139, 208. 

Divergent series, 28, 75. 

Division converted to multiplication, 
516-518. 

Divisor: x is a divisor of y if y mod x = 0 
and x > 0; it is a proper divisor if it is 
a divisor such that 1 < x < y. 

Dixon, Alfred Cardew, 490. 

Dixon, Robert Dan, 509. 

DLINK: Link downward, 409, 411. 

Doran, Robert William, 229. 

Double generating function: A generating 
function of two variables, 94, 396, 

405, 537-539. 

Double order for traversing trees, 332, 

333, 564. 

Doubly linked lists, 280-281, 288-291, 
297-298, 357, 411, 441, 443, 452, 459. 
compared to singly linked, 281, 298. 
Dougall, John, 490. 

Doyle, Arthur Conan, 465. 

Drum memory, 136-137, 457. 

Dull, Brutus Cyclops, 111. 

Dummy variable, 27. 

Dunlap, James Robert, 457. 

Durand, Bruno, 586. 

Dutka, Jacques, 50, 65. 

Dvoretzky, Aryeh (ipsim nnN), 593. 

Dwyer, Barry, 567. 

Dynamic storage allocation, 246-254, 
256-259, 413-414, 435-456. 
history, 457-458, 461-462. 
running time estimates, 449. 

Dynastic order, 336, see Preorder. 

DYSEAC computer, 231. 

e, 23, 619-620, 626. 

Earley, Jackson Clark, 462. 


Easter date, 159-160. 

Edelman, Paul Henry, 598. 

Edge in graph, 363. 

Edwards, Daniel James, 423. 

Effective algorithm, 6, 8, 9. 

Egorychev, Georgii Petrovich (EroptiueB, 
reopmft PleTpoBHu), 499. 

Eisele, Peter, 480. 

Eisenstein, Ferdinand Gotthold Max, 479. 
Elementary symmetric functions, 38, 

94, 497. 

Elevator (lift) system, 282-298. 

Embedding of partial order into linear order, 
262, see Topological sorting. 

Embedding of tree in another tree, 348, 386. 
Emulation, 202. 

END (end of program), 151, 156, 296. 

End of file, 216, 227. 

Endorder, see Postorder. 

Engles, Robert William, 462. 

English letter frequencies, 159. 

ENN1 (enter negative into rll), 133, 210. 

ENNA (enter negative into rA), 133, 210. 

ENNX (enter negative into rX), 133, 210. 

ENT1 (enter into rll), 133, 210. 

ENTA (enter into rA), 133, 210. 

Entity, 233, see Node. 

Entrances to subroutines, 186-191. 
multiple, 189. 

ENTX (enter into rX), 133, 210. 

Enumeration of subtrees, 378-379. 
Enumeration of tree structures, 386-399. 
history, 406-407. 

Epictetus of Hierapolis (’Eitlxiiycoc 
6 TepanoXEcoi;), 1. 

EQU (equivalent to), 146, 149, 155. 
Equivalence algorithm, 360-361, 578, 581. 
Equivalence classes, 354. 

Equivalence declarations, 360. 

Equivalence relations, 353-355, 487. 
Equivalent algorithms, 467. 

Equivalent binary trees, 328-329. 

Equivalent forests, 346. 

Equivalent Lists, 423. 

Equivalent of a MIXAL symbol, 156. 
Equivalent trees, 346. 

Erase a data structure: To return all of its 
nodes to available storage, 
linear list, 273, 274, 279. 

List, 413-414. 

right-threaded binary tree, 333. 

Erdelyi, Arthur, 399. 

Erdwinn, Joel Dyne, 229. 

Errors, avoiding, 260-261, 556. 
computational, 305. 

detection of, 192-193, 201, 257, 297, 413. 
Etherington, Ivor Malcolm Haddon, 399. 
Ettingshausen, Andreas von, 54. 

Etymology, 1-2. 
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Euclid (EuxXei8r]c), 2, 5. 
algorithm for gcd, 2-9, 80. 
algorithm for gcd, extended, 13-14, 42. 
Euclidean domains, 468. 

Euler, Leonhard (EHjiept., JleoHap.g'b = 
Oftjrep, JleoHapfl), 49, 50, 52, 57, 75, 

76, 87, 111, 374, 407, 472, 496, 536, 600. 
constant 7, 75, 107, 114, 619-620. 
summation formula, 111-116, 120, 123. 
theorem, 42. 

totient function <p(n), 42, 184. 

Eulerian trails in a directed graph, 
374-376,379,584. 
enumeration of, 380. 

Eulerian numbers, second-order, 506. 
Evaluation of powers, 509. 

Evaluation of tree functions, 351, 361. 

Evans, Arthur, Jr., 202. 

Exchange operation (-H-), 3, 182, 274. 
Exclusive or, 442, 455, 553. 

Execution time, methods for studying, 

96, 170-172. 

for MIX instructions, 138-141. 

Exercises, notes on, xv-xvii, 284. 

Exit: The place where control leaves 
a routine. 

Exits from subroutines, 186-191. 

Expected value of a random variable: The 
average or “mean” value, 98, 103. 
from a generating function, 100-103. 
Exponential generating function for (a n ): 
J2 a nZ n /n\, 89. 

Exponential integral f?i(x), 498. 

Exponents, laws of, 22, 25, 52. 

Extended binary trees, 399-406. 

Extended Euclidean algorithm, 13-14, 42. 
Extendible matrix, 307. 

External nodes, 400-405. 

External path length, 400, 405. 

Extreme and mean ratio, 80. 

Faa di Bruno, Francesco, 483. 

Factorials, 46-52, 55. 

related to gamma function, 49. 

Factorial powers, 50, 52, 67, 71, 109-110. 
FADD (floating add), 306. 

Fail-safe program, 270. 

Fallacious reasoning, 18, 111, 465. 

Falling powers, 50, 67, 69, 624. 

Family order, 350, 577. 

sequential representation of trees, 350. 
Family trees, 310-311, 317, 406. 

Farber, David Jack, 461. 

Farey, John, 520. 
series, 161. 

Father, in a tree structure, 311. 

FCMP (floating compare), 507, 559. 

FDIV (floating divide), 306. 

Ferguson, David Elton, 231, 334. 


Fermat, Pierre de, 17, 466. 
theorem, 41. 

Ferranti Mark I computer, 18. 

Feynman, Richard Phillips, 26. 

Fibonacci, Leonardo, of Pisa (= Leonardo 
filio Bonacii Pisano), 79-80, 84. 
buddy system, 455. 
generating function, 82-83. 
number system, 86, 495. 
numbers F n : Elements of the Fibonacci 
sequence, 13, 18, 79-86. 
numbers, table of, 621. 
sequence, 13, 18, 79-86, 621. 
strings, 86. 
trees, 496. 

Fibonomial coefficients, 85, 484, 499. 

Fich, Faith Ellen, 523. 

Field: A designated portion of a set of 
data, usually consisting of contiguous 
(adjacent) symbols, 
partial, of MIX word, 126-128, 139, 

143, 207. 

within a node, 233-237. 

within a node, notations for, 235-237, 458. 

FIFO, 240, 459, see Queue. 

Fifty-percent rule, 444-445, 447, 448. 

Figurate numbers, 54. 

Filters, 198. 

Final vertex of an arc, 372. 

Fine, Nathan Jacob, 484. 

First-fit method of storage allocation, 
436-438, 453-456, 616. 

First in, first out, 240, 351, 459, 607, 
see Queue. 

Fischer, Michael John, 353. 

Fixed element of permutation, 164, 180-181. 

Fixed point arithmetic, 158. 

Flag, see Sentinel. 

Flajolet, Philippe Patrick Michel, 501, 

506, 543, 565. 

Fletcher, William, 527. 

Floating point arithmetic, 131, 306. 
operators of MIX, 131, 557-559. 

Floor function |_zj, 39-41. 

Flow charts, viii, 2-3, 15-18, 364-365, 377. 

Floyd, Robert W, x, 17, 20, 422, 467, 509. 

FLPL subroutine library, 460-461. 

Flye Sainte-Marie, Camille, 584. 

FMUL (floating multiply), 306. 

FOCS: Proceedings of the IEEE Symposia 
on Foundations of Computer Science 
(1975-), formerly called the Symposia 
on Switching Circuit Theory and 
Logic Design (1960-1965), Symposia 
on Switching and Automata Theory 
(1966-1974). 

Forder, Henry George, 597. 

Forecasting, 224. 
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Forest: Zero or more trees, 309, 408, 
see Trees. 

correspondence to binary trees, 

334-335, 346. 
enumeration, 389, 594. 
index notation for, 313, 315, 317. 
Formulas, algebraic, see Algebraic formulas. 
Forstemann, Wilhelm August, 490. 
FORTRAN language, viii, 231, 233, 296, 

360, 458, 460. 

Foster, Frederic Gordon, 100. 

Fourier, Jean Baptiste Joseph, 27. 
Fractional part, 40. 

Fraenkel, Aviezri S (i>pns niy'IN), 251. 
Fragmentation, 439, 449, 450, 456. 

Fredman, Michael Lawrence, 514. 

Free lattice, 347. 

Free storage, see Available space. 

Free subtrees, 365-370. 
enumeration of, 378-379. 
minimum cost, 371. 

Free trees, 363-371. 
definition of, 363. 
enumeration of, 387-388, 398, 407. 
Friedman, Daniel Paul, 421. 

Frieze patterns, 407-408. 

Front of queue, 241. 

FSUB (floating subtract), 306. 

Fuchs, David Raymond, 202. 

Fukuoka, Hirobumi (iSlBjtll^), 508. 
Full-word logical (bitwise) operations, 

442, 455, 510, 553. 

Fundamental cycles in a graph, 

366-370, 377. 

Fundamental path, 368. 

Fundamental theorem of arithmetic, 42. 
Furch, Robert, 121. 

Future reference in MIXAL, 153, 156. 
restrictions on, 156. 

Galler, Bernard Aaron, 353. 

Galton, Francis, 562, 633. 

Games, solution of, 86, 272-273. 

Gamma function T(z), 49-52, 72, 79, 
116-119. 

incomplete, 117-122. 

Gao, Zhicheng (jfSiuAjjg), 565. 

Garbage collection, 257, 413-423, 438-439, 
449, 455, 461, 546, 551. 
efficiency of, 420-421. 

Gardner, Martin, 19, 80, 587. 

Garwick, Jan Vaumund, 248, 457. 

Gaskell, Robert Eugene, 86. 

Gasper, George, Jr., 490. 

Gates, William Henry, III, xi. 

Gaufi (= Gauss), Johann Friderich Carl (= 
Carl Friedrich), 49, 58, 95, 490. 
gcd: Greatest common divisor. 

Gelernter, Herbert Leo, 460. 


Generating functions, 82-84, 87-96, 243, 
386-389, 391-392, 394-399, 539, 542. 
double, 94, 396, 405, 537-539. 
for discrete probability distributions, 
99-107, 181. 

Genuys, Frangois, 231. 

Geometric progression, sum of, 31, 88. 
Gerberich, Carl Luther, 460. 

Gill, Stanley, 229, 230, 457. 

Girard, Albert, 497. 

Glaisher, James Whitbread Lee, 504. 

Glassey, Charles Roger, 406. 

Gnedenko, Boris Vladimirovich (T He^reHKO, 
Bopnc BjraflHMnpoBHu), 105. 

GO button of MIX, 126, 139, 143, 211. 
Goldbach, Christian, 49, 407, 472. 

Goldberg, Joel, 528. 

Golden ratio, 13, 18, 21, 80, 83 86, 619-620. 
Goldman, Alan Joseph, 589. 

Goldstine, Herman Heine, 18, 229, 231. 
Golomb, Solomon Wolf, 184. 

Golumbic, Martin Charles 

(P’3di7p orroN p >33 Dran), 596. 
Goncharov, Vasilii Leonidovich (ToHuapoB, 
Bacmmii JleoHHflOBHu), 501. 

Gonnet Haas, Gaston Henry, 395. 

Good, Irving John, 374, 395, 483, 584. 
Gopala (iffTTeT), 80. 

Gorn, Saul. 460. 

Gosper, Ralph William, Jr., 65. 

Gould, Henry Wadsworth, 58, 63, 

121, 485, 492. 

Gourdon, Xavier Richard, 525. 

Gower, John Clifford, 459. 

Grabner, Peter Johannes, 506. 

Graham, Ronald Lewis, 11, 631. 

Graphs, 363-372, 464. 

directed, see Directed graphs. 

Greatest common divisor, 2-9, 13-14, 

40, 81-82. 

Greatest integer function, see Floor function. 
Greek mathematics, 2, 19, 54, 593. 

Griswold, Ralph Edward, 461. 

Grounded wire symbol, 234. 

Griinbaum, Branko, 384, 587. 

Gustavson, Fred Gehrung, 305. 

Guy, Richard Kenneth, 19, 80, 273, 600. 

H-trees, 563. 

Haddon, Bruce Kenneth, 614. 

Hadeler, Karl-Peter Fritz, 480. 

Hageman, Louis Alfred, 586. 

Ilalayudha (f-pTPpr), 53. 

Hald, Anders, 103. 

Hamel, Georg, 480. 

Hamilton, William Rowan, cycles, 374, 379. 
Hamlet, Prince of Denmark, 232. 

Hamming, Richard Wesley, 26. 

Hankel, Hermann, 49. 

Hansen, James Rone, 460. 

Hansen, Wilfred James, 421. 
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Haralambous, Yannis (XapaA<x|xjtou?, 
Tcnawpc), 650. 

Harary, Frank, 407. 

Hardware-oriented algorithms, 26, 252, 611. 
Hardy, Godfrey Harold, 12, 406, 492, 520. 
Hare, David Edwin George, 395. 

Hare and hounds, see Military game. 
Harmonic numbers H n , 75-79, 114. 
generating function, 90. 
table, 621-622. 

Harmonic series, 75, 160-161. 

Haros, Charles, 520. 

Hartmanis, Juris, 464. 

Hautus, Matheus Lodewijk Johannes, 489. 
hcf, see gcd. 

Head of list, see List head. 

Heap, 435, see Pool of available nodes. 
Height of tree or forest, 565. 

Heine, Heinrich Eduard, 490. 

Hellerman, Herbert, 459. 

Hemacandra, Acarya (3TTTPT -4 ) , 80. 
Henkin, Leon Albert, 17. 

Henrici, Peter Karl Eugen, 88. 

Herbert, George, xiv. 

Hermite, Charles, 49, 478. 

Hesse-Kassel, Louise Wilhelmine Friederike 
Karoline Auguste Julia von, 310, 311. 
Heyting, Arend, 406. 

Hilbert, David, matrix, 38. 

Hiles, John Owen, 421. 

Hill, Robert, 518. 

Hindu mathematics, 53-54, 80, 495. 
Hipparchus of Nicasa ("iTraapxo? 

6 lx Ntxala?), 592. 

HLT (halt), 136, 143. 

Hoare, Charles Antony Richard, 17, 

230, 461, 462. 

Hobbes, Thomas, 650. 

Hobby, John Douglas, 650. 

Hofri, Micha (’IDn OD’O), 515. 

Holmes, Thomas Sherlock Scott, 465. 

Holt Hopfenberg, Anatol Wolf, 460. 
Honeywell H800, 124. 

Hopcroft, John Edward, 560. 

Hopper, Grace Brewster Murray, 229. 
Horning, James Jay, 230. 

Hu, Te Chiang 405, 596. 

Huang Bing-Chao (jfJRJi), 176. 

Huffman, David Albert, 402, 407. 

algorithm, 402-406. 

Hurwitz, Adolf, 44. 

binomial theorem, 399, 488. 

Hwang, Frank Kwangming ( jlf yfc Bf] ) , 

405, 595. 

Hwang, Hsien-Kuei (JtcilHI'), 66. 
Hypergeometric functions, 65. 

basic, 490. 

Hyperfactorial, 116. 


I/O: Input or output, 215. 

Il-register of MIX, 125, 142. 

IBM 650 computer, i, 124, 230, 529. 

IBM 701 computer, 230. 

IBM 705 computer, 230. 

IBM 709 computer, 124, 529. 

IBM 7070 computer, 124. 

Ibn al-Haytham, Abu ‘All al-Hasan (= 
Alhazen, A.jll I I ^J), 162. 

Identity permutation, 164, 175. 

Iliffe, John Kenneth, 462. 

ILLIAC I computer, 230. 

Imaginary part of complex number, 21. 
d’lmperio, Mary Evelyn, 462. 

IN (input), 137, 215-216. 

In-degree of vertex, 372. 

in situ permutation, 9, 165, 185, 523. 

INC1 (increase rll), 133, 210. 

INCA (increase rA), 133, 210. 

Incidence matrix, 270. 

Inclusion and exclusion principle, 181, 184. 
Incomplete gamma function y(a, x), 117-122. 
INCX (increase rX), 133, 210. 

Indentation, 312. 

Index: A number that indicates a particular 
element of an array (often called a 
“subscript”), 4, 299, 313, 315. 

Index register, 125, 127, 158, 266. 
modification of MIX instructions, 

127, 251-252. 

Index variable, 27. 

Indian mathematics, 53-54, 80, 495. 

Indirect addressing, 246, 251-252, 306. 
Induction, mathematical, 11-21, 32, 

316, 475. 
generalized, 20. 

Inductive assertions, 15-20. 

Inductive closure, 475. 

Infinite series: A sum over infinitely many 
values, 27-29, 58, 87-96. 

Infinite trees, 317, 382. 

Infinity lemma, 382-386. 

Information: The meaning associated with 
data — the facts or concepts represented 
by data; often used also in a narrower 
sense as a synonym for “data”, or in a 
broader sense to include any concepts 
that can be deduced from data. 
Information structure, see Data structure. 
Ingalls, Daniel Henry Holmes, 522. 

Initial vertex of an arc, 372. 

Inorder for a binary tree, 319-323, 

330-332, 346. ' 

Input, 5, 215-228. 

anticipated, 159, 216, 224. 
buffering, 159, 216-228, 231. 
operators of MIX, 136-138, 215-216. 
Input-restricted deque, 239-243, 416. 
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Insertion of a node: Entering it into 
a data structure. 

into available space list, see Liberation. 

into deque, 251, 297. 

into doubly linked list, 281, 290, 297. 

into doubly linked ring structure, 358. 

into linear list, 239. 

into linked list, 235, 255, 276, 305. 

into quadruply linked binary tree, 333. 

into queue, 242, 244-245, 254, 260, 

265, 273-274. 

into threaded binary tree, 327, 332. 
into two-dimensional list, 305. 
onto a stack, 241, 242, 244-245, 247, 254, 
258, 269, 273-274, 278, 458. 

Instruction, machine language: A code 

that, when interpreted by the circuitry 
of a computer, causes the computer 
to perform some action, 
in MIX, 127-144. 
symbolic form, 128, 144. 

INT (interrupt), 228. 

Integers, 21. 

Integration, 90. 

by parts, 77, 112-113. 
related to summation, 111-116. 
Interchange operation («->), 3, 182, 274. 
Interchanging the order of summation, 

29, 33, 35, 43. 

Interest, compound, 23-24. 

Interlock time: Delay of one part of a 
system while another part is busy 
completing some action. 

Internal nodes, 400-406. 

Internal path length, 400, 402, 405. 

Internet, iv, xvi. 

Interpreter (interpretive routine), 

200-202, 230, 340. 

Interrupt, 228. 

Intervals, notation for, 21. 

Invariants, 17. 

Inverse modulo m, 42. 

Inverse of a matrix, 37-38, 73, 307. 

Inverse of a permutation, 106, 175-178, 182. 
Inversion problem, 63-64. 

Inversions of a permutation, 542, 557, 577. 
Inverting a linked list, 269, 279. 

I/O: Input or output, 215. 

IOC (input-output control), 137. 

IPL systems, 230, 458-459, 460-461, 552. 
Irrational radix, 86. 

Irreflexive relation, 261. 

Isaacs, Irving Martin, 601. 

Islamic mathematics, 1, 162. 

Isolated vertex, 374. 

Itai, Alon (>JT>N fl^N), 534. 

Iverson, Kenneth Eugene, 33, 39, 459-460. 
convention, 32-33, 61, 103. 


J-register of MIX, 125, 143, 186, 189, 
212-214. 

JIN (jump if rll negative), 135, 210. 

JINN (jump if rll nonnegative), 135, 210. 
J1NP (jump if rll nonpositive), 135, 210. 
J1NZ (jump if rll nonzero), 135, 210. 

J1P (jump if rll positive), 135, 210. 

J1Z (jump if rll zero), 135, 210. 

JACM: Journal of the ACM , a publication 
of the Association for Computing 
Machinery since 1954. 

Jacob, Simon, 81. 

Jacobi, Carl Gustav Jacob, 490. 

Jacquard, Joseph Marie, 229. 

Jacquet, Philippe Pierre, 515. 

JAN (jump if rA negative), 135, 210. 

JANN (jump if rA nonnegative), 135, 210. 
JANP (jump if rA nonpositive), 135, 210. 
JANZ (jump if rA nonzero), 135, 210. 

JAP (jump if rA positive), 135, 210. 

Japanese mathematics, 112, 115. 

Jarden, Dov (yrp 31), 85, 494. 

Java language, 233. 

JAZ (jump if rA zero), 135, 210. 

JBUS (jump if busy), 137, 157, 212, 216. 

JE (jump if equal). 135, 209. 

Jeffrey, David John, 395. 

Jenkins, David Philip, 460. 

JG (jump if greater), 135, 209. 

JGE (jump if greater or equal), 135, 209. 

JL (jump if less), 135, 209. 

JLE (jump if less or equal), 135, 209. 

JMP (jump), 134, 187, 209, 288. 

JNE (jump if not equal), 135, 209. 

JN0V (jump if no overflow), 134, 142, 209. 
Jodeit, Jane Griffin, 462. 

Johnson, Lyle Robert, 459-460. 

Johnstone, Mark Stuart, 452. 

Jokes, 54, 200. 

Jones, Clifford Bryn, 18. 

Jones, Mary Whitmore, 378. 

Jonkers, Henricus (= Hans) Bernardus 
Maria, 614. 

Jordan, Marie Ennemond Camille, 388, 406. 
Josephus, Flavius, son of Matthias 

(rvnnn p qt>V = <FXoc(5io{ ’IciariTtoc 
MoaOIou), problem, 162, 184. 

J0V (jump if overflow), 134, 142, 209. 

Joyal, Andre, 395. 

JRED (jump if ready), 137, 222-223. 

JSJ (jump saving rj), 134, 189, 210, 531. 
Jump operators of MIX, 134-135, 209. 

Jump trace, 214, 296, 528. 

JXN (jump if rX negative), 135, 210. 

JXNN (jump if rX nonnegative), 135, 210. 
JXNP (jump if rX nonpositive), 135, 210. 
JXNZ (jump if rX nonzero), 135, 210. 

JXP (jump if rX positive), 135, 210. 

JXZ (jump if rX zero), 135, 210. 
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Kahn, Arthur Bertram, 268. 

Kahrimanian, Harry George, 459. 

Kallick, Bruce, 404. 

Kaplansky, Irving, 184. 

Karamata, Jovan, 66. 

Karp, Richard Manning, 406. 

Katz, Leo, 590. 

Kaucky, Josef, 63. 

Keller, Helen Adams, 123. 

Kepler, Johannes, 80, 81. 

Kilmer, Alfred Joyce, 232. 

King, James Cornelius, 20. 

Kinkelin, Hermann, 504. 

Kirchhoff, Gustav Robert, 406, 583. 

law of conservation of flow, 97, 170-171, 
268, 278, 364-370, 380. 

Kirkman, Thomas Penyngton, 408. 
Kirschenhofer, Peter, 506. 

Klarner, David Anthony, 86. 

Kleitman, Daniel J (Isaiah Solomon), 

547, 596. 

Knopp, Konrad Hermann Theodor, 48, 498. 
Knotted lists, 459. 

Knowlton, Kenneth Charles, 462. 

Knuth, Donald Ervin ii, iv, xi, 11, 

33, 66, 120, 193, 201, 202, 296, 297, 395, 
457, 461, 471, 484, 499, 504, 523, 525, 
565, 579, 580, 584, 592, 596, 631, 650. 
Knuth, Nancy Jill Carter (jfjitfllitf), x, xx. 
Kolmogorov, Andrei Nikolaevich 

(KojiMoropoB, AHApefi HHKOjiaeBHu), 
104, 105, 464. 

Konig, Denes, 382, 406, 588. 

Koster, Cornelis (= Kees) Hermanus 
Antonius, 461. 

Kozelka, Robert Marvin, 544. 

Kramp, Christian, 49, 486. 

Krattenthaler, Christian, 39. 

Kreweras, Germain, 598. 

Krogdahl, Stein, 616. 

Kronecker, Leopold, delta notation, 

33, 61, 624. 

Kruskal, Joseph Bernard, Jr., 386, 588. 
Kummer, Ernst Eduard, 70. 

Kung, Hsiang Tsung 563. 

Labeled trees, enumeration of, 389, 407. 
Labelle, Gilbert, 395. 

Lagrange (= de la Grange), Joseph 
Louis, Comte, 
inversion formula, 392, 594. 

Lame, Gabriel, 407. 

Lamport, Leslie B., 605. 

Language: A set of strings of symbols, 
usually accompanied by conventions 
that assign a “meaning” to each string 
in the set, 5, 241, 460-461. 
machine, viii-x, 124. 


Laplace (= de la Place), Pierre Simon, 
Marquis de, 87. 
transform, 94. 

Laplacian of graph, 582. 

Lapko, Olga Georgievna (JIanKO, Ojitra 
IeoprHeBHa), 650. 

Large programs, writing, 191-193. 

Larus, James Richard, 369. 

Last in, first out, 240, 452, 459, see Stack. 

almost, 446, 449, 455. 

Latency, 228, 457. 

Lattice: An algebraic system that 

generalizes operations like U and n. 
defined on forests, 577, 598. 
free, 347. 

Laurent, Paul Mathieu Hermann, series, 473. 
Lawson, Harold Wilbur, Jr., 433, 461. 

LCHILD field, 352-353, 359-360. 

LD1 (load rll), 129, 208. 

LD1N (load rll negative), 129, 208. 

LDA (load rA), 129, 208. 

LDAN (load rA negative), 129, 208. 

LDX (load rX), 129, 208. 

LDXN (load rX negative), 129, 208. 

Leaf of tree, 308, see Terminal node. 
Least-recently-used replacement, 452. 
Leeuwen, Jan van, 596. 

Left subtree in binary tree, 312, 318. 
Left-to-right maximum or minimum, 

97-101, 104-106, 179. 

Legendre (= Le Gendre), Adrien Marie, 

48, 49, 51. 
symbol, 45. 

Leger, Emile, 80. 

Lehiner, Derrick Henry, 465. 

Leibniz, Gottfried Wilhelm, Freiherr 
von, 2, 51. 

Leighton, Frank Thomson, 450-451. 

Leiner, Alan Lewine, 231. 

Leonardo of Pisa, 79-80, 84. 

Leroux, Pierre, 395. 

Letter frequencies in English, 159. 

Level of node in a tree, 308, 316, 317. 

Level order, 351, 564, 577. 

sequential representation, 351, 359. 
LeVeque, William Judson, 466. 

Levy, Paul, 105. 

Levy, Silvio Vieira Ferreira, xi. 

Lexicographic order, 20, 299-300, 306, 564. 
Liberation of reserved storage, 256, 259, 

291, 413-414, 420-421, 438-442, 
443-444, 452-456. 

LIFO, 240, 459, see Stack. 

Lilius, Aloysius, 159. 

Lindstrom, Gary Edward, 567-568. 

Line printer, 136-137. 

Lineal chart, 310—311. 

Linear extensions, see Topological sorting. 
Linear lists, 232, 238-307. 
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Linear ordering, 20, 262, 270. 

embedding a partial ordering into, 262, 
see Topological sorting, 
of binary trees, 333. 
of trees, 346. 

Linear probing, 451. 

Linear recurrences, 83, 88. 

Link, 233-237. 
diagram, 234. 

field, purpose, 432-433, 462. 
manipulation, avoiding errors in, 260-261. 
null, 234-235. 

Link variable, 234. 

Linked allocation of tables, 234, 254-256. 
arrays, 301-307. 

contrasted to sequential, 254-256, 

296, 433. 
history, 457-461. 

linear lists, 234-237, 264-266, 269. 
tree structures, 334, 352-357. 

Linked memory philosophy, 255, 435. 

Linking automaton, 463-464. 

Linsky, Vladislav Sergeevich (JIhhckhh, 
BjiaflucjiaB CepreeBHu), 471. 

LISP language, 233, 460-461. 

LISP 2 garbage collector, 614. 

List: Ordered sequence of zero or 
more elements. 

circular, 273-280, 302, 355, 411, 459. 
doubly linked, 280-281, 288-291, 297-298, 
357, 411, 441, 443, 452, 459. 
linear, 232, 238-307. 

of available space, see Available space list. 
List (capital-List) structures, 315-316, 
408-423, 460-462. 
copying, 423. 

diagrams of, 315-317, 408-409. 
distinguished from lists, 233, 411. 
equivalence between, 423. 
notations for, 315-317, 408-409. 
representation of, 409-412, 421. 
sequential allocation, 421. 

List head, in circular lists, 275, 302-303. 
in doubly linked lists, 280-281, 

288-289, 441, 443. 
in Lists, 410, 414, 417. 
in threaded binary tree, 324, 334. 
in threaded trees, 337. 

List processing systems, 233, 412, 460-462. 
Listing, Johann Benedict, 406. 

Literal constants in MIXAL, 150, 156. 

Literate programming, 193. 

Littlewood, John Edensor, 406. 

LLINK: Link to the left, 280-281, 288-291. 
in binary trees, 318, 322, 327, 333, 459. 
in Lists, 411. 

in trees, 338, 348, 359, 380. 

LLINKT, 324. 

Lloyd, Stuart Phinney, 183, 184. 

Loading operators of MIX, 129, 139, 208. 


Loading routine, 144, 271-272. 

L0C, 235-236. 

Local symbols of MIXAL, 150-151, 157. 
Locally defined function in tree, 351, 361. 
Location counter in MIXAL, 155. 

Location field of MIXAL line, 145, 152. 

Logan, Benjamin Franklin (= Tex), Jr., 74. 
Logarithms, 22-26. 
binary, 23, 26. 
common, 23, 26. 
natural, 23, 26. 
power series, 91. 

Loop in a directed graph: Arc from a 
vertex to itself, 372. 

Loopstra, Bram Jan, 231. 

Louise Wilhelmine Friederike Karoline 
Auguste Julia von Hesse-Kassel, 

310, 311. 

Lovasz, Laszlo, 491. 

Lovelace, Augusta Ada Byron King, 
Countess of, 1. 

LTAG, 322, 333, 349-350, 352, 359, 380. 

Lucas, Frangois Edouard Anatole, 69, 

80, 81, 85, 273, 494. 
numbers L n , 495. 

Luhn, Hans Peter, 457. 

Lukasiewicz, Jan, 338. 

Lunch counter problem, 456. 

Luo, Jianjin 407. 

Lushbaugh, Warren Arthur, 19. 

Lynch, William Charles, 585. 

Machine language: A language that directly 
governs a computer’s actions, as 
it is interpreted by the computer’s 
circuitry, viii-x, 124. 
symbolic, 144, see Assembly language. 
MacMahon, Percy Alexander, 490, 589. 
Macro instruction: Specification of a pattern 
of instructions and/or pseudo-operators 
that may be repeated frequently 
within a program. 

Madnick, Stuart Elliot, 461. 

Magic square, 162. 

Magnetic tape, 136-137, 463. 

Mahavira, Acarya (Hi^iql < P3 1 4 ), 54. 
Mailloux, Barry James, 461. 

Majorization, 406. 

malloc, see Dynamic storage allocation. 
Mallows, Colin Lingwood, 536. 

Margolin, Barry Herbert, 450. 

Mark I calculator (Harvard), 229. 

Mark I computer (Manchester/Ferranti), 18. 
Mark bits, 413-414. 

Marking algorithms: Algorithms that 

“mark” all of the nodes accessible from 
some given node, 271-272, 415-423. 
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Markov ( = Markoff), Andrei Andreevich 
(MapKOB, Aipipeii ArpupeeBUH), 
the elder, 495. 

chain: Path taken by a Markov process, 
process, 253, 380-382. 

Markov, Andrei Andreevich (MapKOB, 

AHflpefi An,rpeeBn'{), the younger, 8. 
Markowitz, Harry Max, 461. 

Markowsky, George, 80, 404. 

Martin, Alain Jean, 605. 

Martin, Johannes Jakob, 614. 

Math. Comp.: Mathematics of Computation 
(I960-), a publication of the American 
Mathematical Society since 1965; 
founded by the National Research 
Council of the National Academy 
of Sciences under the original title 
Mathematical Tables and Other Aids 
to Computation (1943-1959). 
Mathematical induction, 11-21, 32, 316, 475. 
generalized, 20. 

Matiyasevich, Yuri Vladimirovich 

(MaTHBCeBHU, 1 ( ) p M H B.JI a,a M M up OKU 'I ) , 
86. 

Matrix: A two-dimensional array, 

298-299, 315. 

Cauchy, 37-38, 475. 
characteristic polynomial of, 499. 
combinatorial, 37-38, 589. 
determinant of, 37-39, 81, 378-379, 382. 
extendible, 307. 

Hilbert, 38. 
incidence, 270. 
inverse of, 37-38, 73, 307. 
multiplication, 306. 
permanent of, 51. 

representation of, 158-159, 298-307. 

singular, 307. 

sparse, 302-306. 

transpose of, 182. 

triangular, 300, 305. 

tridiagonal, 307. 

unimodular, 601. 

Vandermonde, 37-38, 475. 

Matrix (Bush), Irving Joshua, 35, 36. 

Matrix tree theorem, 378-379, 586. 

Mauchly, John William, 230. 

Maurolico, Francesco, 17. 

Maximum, algorithm to find, 96-101, 

145, 186. 

Maximum norm, 106. 

McCall's, v. 

McCann, Anthony Paul, 614. 

McCarthy, John, 460-461. 

McEliece, Robert James, 477. 

Mcllroy, Malcolm Douglas, 576, 581. 
McKeeman, William Marshall, 230. 

Mealy, George, 462. 

Mean value, see Expected value. 

Meek, Homer Vergil, 230. 


Meggitt, John Edward, 471. 

Melville, Robert Christian, 540. 

Memory: Part of a computer system 
used to store data, 126. 
cell of, 127. 

hierarchy, 199, 421, 435, 463. 
map, 435-436. 
types of, 238. 

Merner, Jack Newton Forsythe, 229. 
Merrett, Timothy Howard, 560. 

Merrington, Maxine, 66. 

METHFONT, iv, xi, 611, 650. 

METRPOST, xi, 650. 

Military game, 273, 550. 

Miller, Kenneth William, 123. 

Ming, An-T’u (PJiSll), 407. 

Minimum path length, 399-406. 

Minimum spanning tree, 371. 

Minimum wire length, 371. 

Minsky, Marvin Lee, 423. 

Mirsky, Leon, 587. 

Mitchell, William Charles, 525. 

MIX computer, viii-x, 124-144. 
assembly language for, 144-157. 
extensions to, 143, 228, 251-252, 455. 
instructions, summary, 140-141. 
simulator of, 203-212. 

Mix Barrington, David Arno, 526. 

MIXAL: MIX Assembly Language, 144-157, 
235-236. 

Mixed-radix number system, 300. 

Mixture of probability distributions, 106. 
MMIX computer, 124, 187, 215, 325. 

Mock, Owen Russell, 231. 
mod, 39-40. 
modulo, 40. 

Mohammed, John Llewelyn, 527. 

Moivre, Abraham de, 74, 83, 87, 106, 

182, 474, 536. 

Moments of probability distributions, 105. 
Monitor routine, 212, see Trace routine. 
Monte Carlo method: Experiments with 
random data, 254, 445-447. 

Moon, John Wesley, 407. 

Moore School of Electrical Engineering, 230. 
Mordell, Louis Joel, 479. 

Morris, Francis Lockwood, 18, 614. 

Morris, Joseph Martin, 567. 

Morrison, Emily Kramer, 229. 

Morrison, Philip, 229. 

Moser, Leo, 66. 

Mother, in a tree structure, 311. 

Motzkin, Theodor Samuel 

(VP810 TlTIN’n), 85, 593. 

MOVE, 135, 142, 193, 211. 

MOVE CORRESPONDING, 426, 430-431, 434. 
MUG: MIX User’s Group, 641. 

MUL (multiply), 131-132, 208. 

Multilinked structures, 232, 288-289, 

357, 424-434, 458. 
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Multilist representation, 301. 

Multinomial coefficients, 65, 394. 
Multinomial theorem, 65. 

Multipass algorithms, 198-200, 201-202. 
Multiple: a; is a multiple of y if x = ky 
for some integer k. 

Multiple entrances to subroutines, 189. 
Multiple exits from subroutines, 190, 269. 
Multiple precision arithmetic, 202. 

Multiple precision constants, 619-621. 
Multiple summation, 33-36. 

Multiplication of permutations, 165-167, 
172-173, 371. 

Multiplication of polynomials, 277, 280, 361. 
Multiplication of sparse matrices, 306. 
Multiplicative function, 42-43. 

Multiset: Analogous to a set, but elements 
may appear more than once. 

Multiway decisions, 158. 

Munro, James Ian, 523. 

Nagorny, Nikolai Makarovich (HaropHbiii, 
HnKOjiaii MaKapoBHu), 9. 

Nahapetian, Armen, 579. 

Napier, John, Laird of Merchiston, 23. 
Narayana, Tadepalli Venkata 
(irc?Z& ;ru*d£ora), 598. 

Nash, Paul, 556. 

National Science Foundation, x. 

Natural correspondence between binary 
trees and forests, 334-335, 346. 

Natural logarithms, 23, 26. 

Naur, Peter, 17. 

Needham, Joseph, 59. 

Neely, Michael, 452. 

Negative: Less than zero ( not zero). 
Negative binomial distribution, 107. 

Nested parentheses, 312-313, 349, 597. 
Nested sets, 312, 317. 

Nesting store, 240. 

Network: A graph together with additional 
data, such as weights on the edges 
or vertices. 

Neumann, John von (= Margittai Neumann 
Janos), 18, 229, 231, 457. 

Neville, Eric Harold, 591. 

Newell, Allen, 230, 457-458, 460. 

Newton, Isaac, 22, 57, 497. 
identities, 497. 

Next-fit method of storage allocation, 

448, 453, 618. 

Nicolau, Alexandru, 614. 

Nicomachus of Gerasa ( N ixop-axo? 

6 ix Tepaocov), 19. 

Nielsen, Norman Russell, 450. 

Nil link, see Null link. 

Niven, Ivan Morton, 87. 

Noah, son of Lamech (fa!? p m), 310. 


Node: Basic component of data structures, 
233, 462-464. 
address of, 233. 
diagram of, 234. 
link to, 233. 

notations for fields, 235-237, 458. 
size of, 257, 299, 435, 453. 

NODE, 236. 

Node variable, 236. 

Noncrossing partitions of a polygon, 408. 
Nonnegative: Zero or positive. 

NOP (no operation), 136. 

Normal distribution, 104, 122. 

approximately, 105-106. 

Notations, index to, 623-627. 

Null link (A), 234-235. 
in binary trees, 322, 331. 
in diagrams, 234. 
in trees, 318. 

NUM (convert to numeric), 138. 

Number definitions, 21. 

Number system: A language for representing 
numbers, 
binary, 24-26. 
combinatorial, 73, 560. 
decimal, 21, 691. 

Fibonacci, 86, 495. 
mixed-radix, 300. 
octal, 620. 
phi, 86. 

Number theory, elementary, 40-45. 

Nygaard, Kristen, 229, 461. 

O-notation, 107-111, 118. 

O’Beirne, Thomas Hay, 518. 

Octal values of constants, 620. 

Odlyzko, Andrew Michael, 121, 565. 
Oettinger, Anthony Gervin, 460. 

Office of Naval Research, x, 230. 

Okada, Satio (|3]EB#ffi, later 
582. 

Oldenburg, Henry, 57. 

Oldham, Jeffrey David, xi. 

Omphaloskepsis, 214. 

One-address computer, 127, 350. 

One-way equalities, 108. 

One-way linkage, see Circular linkage, 
Straight linkage. 

Onodera, Rikio (''l'>Sf^'^J^), 582. 

Open subroutine, 229, see Macro instruction. 
Operation code field, of MIX instruction, 127. 

of MIXAL line, 145, 155. 

Optimal search procedure, 402. 

Order of succession to the throne, 336. 
Ordered trees, 308-309, 374, see Trees. 

enumeration of, 388-389, 398, 407. 
Ordering: A transitive relation between 
objects of a set. 

lexicographic, 20, 299-300, 306. 
linear, 20, 262, 270. 
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linear, of tree structures, 333, 346. 
partial, 261-262, 269-270, 346, 562, 575. 
well-, 20, 334. 

Oresme, Nicole, 22. 

Oriented binary trees, 396-397. 

Oriented cycle in a directed graph, 372. 
Oriented forests, 353-355. 

Oriented path in a directed graph, 372. 
Oriented subtrees, enumerated, 378. 
Oriented trees, 308-309, 312, 372-382. 
canonically represented, 390-394, 
397-398, 590-591. 
converted to ordered trees, 347. 
defined, 373. 

enumerated, 386-387, 389-395, 406. 
represented in computer, 347, 353, 377. 
with root changed, 377. 

0RIG (origin), 146, 151, 155. 

Orlin, James Berger, 584. 

Orthogonal lists, 298-307. 

Orthogonal vectors of permutations, 184. 
Otoo, Ekow Joseph, 560. 

Otter, Richard Robert, 395, 589. 

OUT (output), 137, 225. 

Out-degree of a vertex, 372. 

Output, 5, 215-228. 

buffering, 147, 216-228, 231. 
operators of MIX, 136-138. 
Output-restricted deque, 239-243, 269, 274. 
OVERFLOW, 245-251, 256-258, 268-269. 
Overflow toggle of MIX, 126, 134, 142, 

208, 214, 228. 

Overlapping subtrees, 326, 603. 

Packed data: Data that has been 

compressed into a small space, as when 
two or more elements are placed into 
the same cell of memory, 128, 158. 
Paging, 452. 

Pallo, Jean Marcel, 577. 

Palm tree, 581. 

Paper sizes, 318. 

Paper tape, 136-137, 229, 231. 

Parallelism, 296. 

Parameters of subroutines, 187, 189, 229. 
Parent, in a tree structure, 311, 317, 
334-335. 

in a threaded tree, 566. 

Parent links, 347, 353-355, 359-361, 

373, 377, 427-433. 

Parentheses, 312-313, 349, 597. 

Parker, Douglass Stott, Jr., 596. 

Parmelee, Richard Paine, 450. 

Partial field designations in MIX. 126-128, 
139, 143, 207. 

Partial fractions, 62-63, 72, 83. 

Partial ordering, 261-262, 269-270, 

346, 562, 575. 

Partitions of a set, 74, 482. 

Partitions of an integer, 12, 34, 93. 
generating function, 87, 94. 


Pascal, Blaise, 17, 53. 
triangle, 53-54, 69, 71, 73, 85, 499, see 
Binomial coefficients. 

Pass, in a program, 198-200. 

Patashnik, Oren, 11, 631. 

Path, in a graph or directed graph, 363. 
oriented, 372. 
random, 380 -381. 
simple, 363, 369, 372, 376. 

Path compression, 576. 

Path length of a tree structure, 399-406. 
average, 405. 

Patience (solitaire), 377-378. 

Patt, Yale Nance, 509. 

Patterns in permutations, 243. 

Pawlak, Zdzislaw, 460. 

PDP-4 computer, 124. 

Peck, John Edward L., 461. 

Pedigree, 310-311. 

Peirce, Charles Santiago Sanders, 593. 
Penrose, Roger, 587. 

Peripheral device: An I/O component of 
a computer system, 136. 

Perlis, Alan Jay, 322, 459-460. 

Permanent of a square matrix, 51. 
Permutations, 45-46, 51, 97-98, 164-185, 
242-243. 

in place, 9, 165, 185, 523. 
inverse of, 106, 175-178, 182. 
multiplication of, 165-167, 172-173, 371. 
notations for, 164. 
orthogonal vectors of, 184. 

Persian mathematics, 1. 

PERT network, 261-262. 

Petkovsek, Marko, 65. 

Petolino, Joseph Anthony, Jr., 516. 

Pfaff, Johann Friedrich, 486. 

Pflug, Georg Christian, 445. 

Phi (</>), 81, see Golden ratio. 

number system, 86. 

Phidias, son of Charmides (4>si5[a; 
Xap[ju5ou), 81. 

Philco S2000 computer, 124. 

Phyllotaxis, 80. 

Pi (tt), 21, 619-620. 

as “random” example, 397. 

Wallis’s product for, 52, 116. 

Pingala, Acarya (34Nl4 fH-jj-M ), 53. 

Pile, 240. 

Pilot ACE computer, 230. 

Pipe, 198. 

Pipeline, 528. 

Pisano, Leonardo, 79-80, 84. 

Pivot step, 302-305, 307. 

PL/I language, 433-434. 

PL/MIX language, 156. 

Plane trees, 308, see Ordered trees. 

Plex, 458. 

Poblete Olivares, Patricio Vicente, 523. 
Poincare, Jules Henri, 491. 
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Pointer, see Link. 

Pointer machines, 464. 

Poirot, Hercule, xvii. 

Poisson, Simeon Denis, distribution, 

106, 524. 
tail of, 502. 

Polish notation, 338, see Prefix notation, 
Postfix notation. 

Polonsky, Ivan Paul, 461. 

Polya, Gyorgy (= George), 17, 93, 395, 

396, 406, 407, 496. 

Polynomials, 55, 57, 64, 67, 68, 70, 108. 
addition of, 275-280, 357-359. 

Bernoulli, 44, 113-115, 503. 
differences of, 64. 
multiplication of, 277, 280, 361. 
representation of, 275-276, 280, 356-357. 
Pool of available nodes, 257, see Available 
space list. 

Pooled buffers, 224, 227. 

Pop up a stack: Delete its top element, 

241, 242, 244-245, 247, 254, 259, 

269, 273-274, 278, 458. 

Positive: Greater than zero (not zero). 
Postfix notation, 338, 352, 593. 

Posting a new item, see Insertion. 

Postorder for a binary tree, 319, 321, 
330-332, 346. 

Postorder for a tree, 336-340, 346, 348, 460. 
Postorder with degrees, representation 
of trees, 351, 361-362. 

PostScript language, 202. 

Poupard, Yves, 598. 

Power of a number, 22. 
evaluation, 509. 

Power series: A sum of the form J2k>0 a k zk < 
see Generating function, 
convergence of, 87, 396. 
manipulation of, 118. 

Pratt, Vaughan Ronald, 45, 539, 592. 

Prefix notation, 338. 

Preorder for a binary tree, 319, 321, 

330-332, 346. 

Preorder for a tree, 336—338, 346, 348, 460. 
Preorder sequential representation of 
trees, 349, 362. 
with degrees, 359, 460. 

Prepostorder, 568. 

Prim, Robert Clay, 371. 

Prime numbers, 19, 41, 45, 47-48, 51, 

69-70, 84-85. 

algorithm to compute, 147-149. 
factorization into, 42. 

Prinz, Dietrich Giinter, 230. 

Priority queue, 435, 556, 590. 

Probability distribution: A specification of 
probabilities that govern the value of 
a random variable, 98-107. 
average (“expected”) value of, 98-103. 
variance of, 98-103. 


Probability generating function, 103. 

Procedure, see Subroutine. 

Procedure for reading this set of books, 
xii-xiv, 9. 

Prodinger, Helmut, 506. 

Profile of a program: The number of 
times each instruction is performed, 

145, 170, 214, 296, 528. 

Program: Representation of a computational 
method in some precise, formalized 
language, 5. 

Programming language: A precise, 

formalized language in which programs 
are written. 

Programs, hints for construction of, 

191-193, 296. 

Progression, arithmetic, sum of, 11, 

31-32, 56. 

Progression, geometric, sum of, 31, 88. 

Proof of algorithms, 5-6, 13-17, 321, 

361, 422, 434. 

Proof of termination, 16-17, 19-21, 386. 

Proper divisor, see Divisor. 

Property A , 586. 

Prosody, 53, 80. 

Priifer, Ernst Paul Heinz, 407. 

Pseudo-operator: A construction in a 

programming language that is used to 
control the translation of that language 
into machine language, 146. 

Psi function ip(z), 44, 75, 493. 

Purdom, Paul Walton, Jr., 448, 450. 

Push down list, 240, see Stack. 

Push down a stack: Insert a new top 

element, 241, 242, 244-245, 247, 254, 
258, 269, 273-274, 278, 458. 

g-nomial coefficients, 65, 73, 484, 491. 

g-nomial theorem, 73, 494. 

Quadratic Euclidean domains, 468. 

Quadratic reciprocity law, 45. 

Quadruply linked binary tree, 333. 

Quadruply linked trees, 357. 

Quadtrees, 564. 

Qualification of names, 424-434. 

Quasiparallel processing, 296. 

Quasitoroidal tiling, 586. 

Queue, 239-243, 264-266, 459, 577, 607. 
deletion from the front, 242, 244-245, 

254, 261, 265, 273-274. 
insertion at the rear, 242, 244-245, 

254, 260, 265, 273-274. 
linked allocation, 259-261, 269, 

273-274, 288. 

sequential allocation. 244-245, 251, 

252, 254. 

Quick, Jonathan Horatio, 502. 

Quotient, 40. 
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Rahman, Mizanur (Ri'ni-Jt 490. 

Railway network, 240. 

Ramanan, Prakash Viriyur (dilrr&rrap 
sSlifl^rr ywsmm ) , 538. 

Ramanujan Iyengar, Srinivasa (uT$®sflsvrTsrv 
rrrrwrTSQienm ggiumamf), 12, 121, 122. 
Ramshaw, Lyle Harold, 487. 

Ramus, Christian, 71. 

Randeil, Brian, 202, 450. 

Random path, 380-381. 

Raney, George Neal, 392, 394, 593. 

Raphael, Bertram, 460. 

Rational number, 21, 25, 161. 

Raz, Yoav (n nNl’), 534. 

RCA 601 computer, 124. 

Reactive process, 5. 

Read, Ronald Cedric, 565. 

Reading: Doing input, 215. 

Real number, 21. 

Real part of complex number, 21. 

Real-time garbage collection, 423. 
Reallocation of memory, 247-251, 452, 457, 
see also Compacting memory. 

Rear of queue, 241. 

Recently used bit, 452. 

Recipe, 6. 

Reciprocity formulas, 44-45. 

Recomp II computer, 124. 

Record: A set of contiguous data; see 
also Node, 233. 

Records, blocking of, 218, 225. 

Rectangular arrays, 298-307. 

Recurrence relation: A rule that defines each 
element of a sequence in terms of the 
preceding elements, 87-89. 

Recursion induction, 321, 565, 569. 

Recursive definition, 308, 312, 315, 318, 

319, 335, 346, 357. 

Recursive Lists, 316. 

Recursive use of subroutines, 191. 

Redheffer, Raymond Moos, 491. 

Reeves, Colin Morrison, 445. 

Reference, 233, see Link. 

Reference counters, 413-414, 421, 461. 
Reflection principle, 536, 593, 598. 

Reflective laws, 55, 489. 

Reflexive relation, 261, 353. 

Registers: Portions of a computer’s 
internal circuitry in which data is 
most accessible, 
of MIX, 125. 

saving and restoring contents of, 188, 

198, 213, 228. 

Regular directed graph, 379. 

Reingold, Edward Martin ("Dupn, 
o«n p nwa pn:p), 23, 518. 

Relation: A property that holds for certain 
sets (usually ordered pairs) of elements; 
for example, “<” is a relation defined 
on ordered pairs (x,y) of integers, 


and the property “x < y ” holds if and 
only if x is less than y. 
antisymmetric, 261. 
asymmetric, 261. 
equivalence, 353-355, 487. 
irreflexive, 261. 
reflexive, 261, 353. 
symmetric, 353. 

transitive, 108, 261, 353, see Ordering. 

Relative error, 116. 

Relatively prime integers, 40. 

Releasing a buffer, 219-223, 226-227. 

Remainder, 40, 160. 

Remove from a structure, see Deletion. 

Renyi, Alfred, 595. 

Replacement operation (•<—), 3. 

Replicative function, 43-44. 

Representation (inside a computer), 
methods for choosing, 238, 424-433. 
of algebraic formulas, 337, 459. 
of binary trees, 318, 322, 327, 333-334. 
of deques, 251, 280, 297. 
of forests, 334. 
of Lists, 409-412, 421. 
of oriented trees, 347, 353, 377. 
of polynomials, 275-276, 280, 356-357. 
of queues, 244-245, 251-254, 259-261, 
269, 273-274, 288. 
of stacks, 244-254, 258, 269-270, 
273-274, 332, 417. 
of trees, 348-357, 359-362, 459-460. 

Reprogramming, 203. 

Reservation of free storage, 256-258, 291, 
436-438, 443, 452-456. 

Reversal of a string, 185. 

Reversing a list, 269, 279. 

Reversion storage, 240. 

Ribenboim, Paulo, 466. 

Rice, Stephan Oswald, 565. 

Richmond, Lawrence Bruce, 565. 

Riemann, Georg Friedrich Bernhard, zeta 
function £(s), 76, 504. 

Right subtree of a binary tree, 312, 318. 

Right-threaded binary trees, 327, 332-334. 

Right-threaded trees, 338, 380. 

Right-to-left maximum or minimum, 
97-101, 104-106, 179. 

Ring structure, 355-357. 

Riordan, John, 397, 494, 595. 

RISC: Reduced Instruction Set 
Computer, 124. 

Rising factorial powers, 50, 71, 624. 

Ritchie, Dennis MacAlistair, 461. 

RLINK: Link to the right, 280-281, 288-291. 
in binary trees, 318, 322, 327, 333, 459. 
in Lists, 409, 411. 
in trees, 338, 348-353, 359, 380. 

RLINKT, 324. 

Robinson, Raphael Mitchel, 587. 
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Robson, John Michael, 448, 452, 456, 

567, 604, 616. 

Rodrigues, Benjamin Olinde, 407. 

Roes, Piet Bernard Marie, 100. 

Rogers, Leonard James, 490. 

Rokicki, Tomas Gerhard, 202. 

Roll, 240. 

Root of a directed graph, 372. 

Root of a number, 22, 25, 110. 

Root of a tree, 308, 309, 317. 

changing, in an oriented tree, 377. 

Rooting a free tree, 373. 

Roots of unity, 89. 

Rosenberg, Arnold Leonard, 560. 

Rosenstiehl, Pierre, 243. 

Ross, Douglas Taylor, 450, 458, 462. 

Rotating memory devices, 228, 457. 

Rothe, Heinrich August, 63, 71, 486. 
Rounding, 41, 83, 160, 183. 

Rousseau, Cecil Clyde, 507. 

Roving pointer, 607. 

Row major order, 159, 182, 299. 

RTAG, 322, 332-334, 338, 349-351, 359, 380. 
Running time, see Execution time. 

Russell, David Lewis, 613. 

Russell, Lawford John, 202. 

Saddle point, 159. 

Salton, Gerard Anton, 351, 459. 

Sammet, Jean Elaine, 462, 574. 
Satterthwaite, Edwin Hallowell, Jr., 231. 
Saving and restoring registers, 188, 

198, 213, 228. 

Schaffer, Alejandro Alberto, 514. 

Schatzoff, Martin, 450. 

Scherk, Heinrich Ferdinand, 490. 

Schlatter, Charles Fordemwalt, 459. 
Schlatter, William Joseph, 459. 
Schlesvig-Holstein-Spnderborg-Glucksborg, 
Christian von, see Christian IX. 
Scholten, Carel Steven, 231, 605. 

Schoor, Amir (T\y TON), 559. 

Schorr, Herbert, 418, 422. 

Schreiber, Peter, 81. 

Schreier, Otto, 385. 

Schroder, Friedrich Wilhelm Karl Ernst, 592. 
Schwartz, Eugene Sidney, 404. 

Schwarz, Karl Hermann Amandus, 
inequality, 36. 

Schweins, Franz Ferdinand, 489. 

Schwenk, Allen John, 495. 

Schweppe, Earl Justin, 459. 

SCOPE link, 350, 362, 434. 

Scroll, 240. 

Segner, Johann Andreas von, 407, 536. 

Seki, Takakazu (H##]), 112, 115. 
Self-modifying code, 187, 193. 

Selfridge, John Lewis, 78. 

Semaphores, 231. 


Semi-invariants of a probability distribution, 
103-106. 

Sentinel: A special value placed in a table, 
designed to be easily recognizable 
by the accompanying program, 

217-218, 276, 567. 

Sequential (consecutive) allocation of 
tables, 244. 

arrays, 158-159, 299-301, 305-307. 
contrasted to linked, 254-256, 296, 433. 
history, 457. 

linear lists, 244-254, 264-266, 325. 
tree structures, 348-352, 359-362, 433. 
Series, infinite: An infinite sum. 
Series-parallel networks, 589. 

Sets, partition of, 74, 482. 

Sha, Jichang 547. 

Shakespeare (= Shakspere), William, 

232, 466. 

Shams Baragh, Armin (jl^ 526. 

Shared subtrees, 326, 603. 

Shaw, John Clifford, 230, 457-458. 

Shelf, 240. 

Shephard, Geoffrey Colin, 384, 587. 

Shepp, Lawrence Alan, 183, 184. 

Shift operators of MIX, 135, 211. 

Shor, Peter Williston, 514. 

Shore, John Edward, 445, 450. 

Shylock, 466. 

Sibling, in a tree structure, 311. 

Sibling link, 334, 336, 427-433; see also 
RLINK in trees. 

SICOMP: SIAM Journal on Computing , 
published by the Society for Industrial 
and Applied Mathematics since 1972. 
Sideways addition, 131, 480. 

Sign function (sign#), 475. 

Silver, Roland Lazarus, 527. 

Similar binary trees, 327-329. 

Similar forests, 346. 

Simon, Herbert Alexander, 230, 457-458. 
Simonovits, Miklos, 505. 

Simple oriented path, 372, 376. 

Simple path, 363, 369. 

SIMSCRIPT language, 461. 

SIMULA I language, 229. 

Simulated time, 283, 288. 

Simulation: Imitiation of some prc*:ess, 445. 
continuous, 282, 298. 
discrete, 203, 282-298. 
of one computer on another, 9, 202-203. 
of one computer on itself, 212-214. 

Singh, Parmanand (TT^TRcT fTT^T), 80. 
Singleton cycle of a permutation, 164, 

171, 180-181. 

Singular matrix, 307. 

Singularity of a function, 396. 

Sister, in a tree structure, 311. 

SLA (shift left rA), 135, 530. 

SLAX (shift left rAX), 135, 530. 
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SLC (shift left rAX circularly), 135, 530. 
SLIP system, 460-461. 

Sloane, Neil James Alexander, 595. 

Smallest in, first out, 556. 

SNOBOL language, 461. 

SODA: Proceedings of the ACM-SIAM 
Symposia on Discrete Algorithms, 
inaugurated in 1990. 

Software: General-purpose programs 
that extend the capabilities of 
computer hardware. 

Solitaire (patience), 377-378. 

Son, in a tree structure, 311. 

Soria, Michele, 501. 

Spanning subtrees, 365-370, 378-379. 

minimum cost, 371. 

Sparse array trick, 307. 

Sparse matrices, 302-306. 

Sparse-update memory, 298. 

Speedcoding, 230. 

Spiefi, Jurgen, 91. 

Spine of a binary tree, 568. 

SRA (shift right rA), 135, 530. 

SRAX (shift right rAX), 135, 530. 

SRC (shift right rAX circularly), 135, 530. 
ST1 (store rll), 130, 209. 

STA (store rA), 130, 209. 

Stack, 239-243, 320-321, 323, 325-326, 351, 
361, 415-416, 422, 428-429, 458-459. 
deletion (“popping”), 241, 242, 244-245, 
247, 254, 259, 269, 273-274, 278, 458. 
insertion (“pushing”), 241, 242, 244-245, 
247, 254, 258, 269, 273-274, 278, 458. 
linked allocation, 258, 269, 270, 

273-274, 332, 417. 
pointer to, 244, 258. 
sequential allocation, 244-254, 325. 

Stack permutations, 242-243, 331. 

Standard deviation of a probability 

distribution: The square root of the 
variance, an indication of how much 
a random quantity tends to deviate 
from its expected value, 98. 

Stanford University, ii, x, 296, 554. 

Stanley, Richard Peter, 407, 593, 598. 
Staudt, Karl Georg Christian von, 406. 
Steady state, 381-382. 

Stearns, Richard Edwin, 464. 

Steele, Guy Lewis, Jr., 605. 

Steffens, Elisabeth Francisca Maria, 605. 
Steffensen, Johan Frederik, 503. 

Stevenson, Francis Robert, 579. 
Stickelberger, Ludwig, 51. 

Stigler, Stephen Mack, 448, 450. 

Stirling, James, 47-49, 67, 69, 73, 

87, 115, 181. 

approximation, 51, 115-116. 

Stirling numbers, 66-69, 71-74, 78, 

99-100, 506, 582, 591. 
asymptotic behavior, 66. 


combinatorial interpretations, 66, 74, 179. 

duality law, 68. 

generating functions, 91. 

modulo p, 492. 

table, 66. 

STJ (store rJ), 130, 187, 209. 

STOC: Proceedings of the ACM 

Symposia on Theory of Computing, 
inaugurated in 1969. 

Stolarsky, Kenneth Barry, 495. 

Storage allocation: Choosing memory cells 
in which to store data, see Available 
space list, Dynamic storage allocation, 
Linked allocation, Sequential allocation. 
Storage mapping function: The function 
whose value is the location of an 
array node, given the indices of that 
node, 299-301, 305-307. 

Store: British word for “memory”. 

Storing operators of MIX, 130, 209. 

Straight linkage, 254, 258-259, 411. 

String: A finite sequence of zero or more 
symbols from a given alphabet, 8, 86, 
185, 274, 495, see Linear lists, 
binary, 598-599. 
concatenation, 274. 
manipulation, 461, 462. 

Strong, Hovey Raymond, Jr., 560. 

Strongly connected directed graph, 372, 377. 
Structure, how to represent, 238, 

424-433, 462. 

Struik, Dirk Jan, 57, 481, 497. 

Stuart, Alan, 100. 

STX (store rX), 130, 209. 

STZ (store zero), 130, 209. 

SUB (subtract), 131-132, 208. 

Subadditive law, 616. 

Subi, Carlos Samuel, 527. 

Subroutines, 158, 186-197, 202, 206-207, 
211, 269, 279-280, 290-291, 343. 
allocation of, 271-272. 
history, 229-230. 
linkage of, 186, 229. 

Subscript, 3-4, see Index. 

Substitution operation (<— ), 3. 

Subtrees, 308. 

average size of, 405. 

free, enumeration of, 378-379. 

Summation, 27-39. 
by parts, 44, 76-78. 

Euler’s formula, 111-116, 120, 123. 
interchange of order, 29, 33, 35, 43. 
multiple, 33-36. 

of arithmetic progression, 11, 31-32, 56. 
of binomial coefficients 56-74, 

76-78, 85, 96. 

of geometric progression, 31, 88. 
of powers, 115. 

related to integration, 111-116. 

Sun SPARCstation, 650. 
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Supremum: Least upper bound, 37. 

Suri, Subhash (g^TTT^ft), 514. 

Sutherland, Ivan Edward, 459. 

Swainson, William, 330. 

Swapping buffers, 147, 159, 217-218, 225. 

Swift, Charles James, 231. 

Swift, Jonathan, 628. 

Switching table, 158, 205-206, 209, 210, 530. 

Sylvester, James Joseph, 407, 473, 583, 586. 

Symbol manipulation: A general term 
for data processing, usually applied 
to nonnumeric processes such as 
the manipulation of strings or 
algebraic formulas. 

Symbol table algorithms, 175, 426. 

Symbolic machine language, see Assembly 
language. 

Symmetric functions, 92-94, 497. 
elementary, 38, 94, 497. 

Symmetric order for a binary tree, 

319-323, 330-332, 346. 

Symmetric relation, 353. 

Synchronous discrete simulation, 282, 298. 

System: A set of objects or processes 
that are connected to or interacting 
with each other. 

System/360 computers, 124, 189, 230. 

Szekeres, George, 595. 

Szpilrajn, Edward, 268. 

i-ary trees, 334, 405. 

enumeration of, 397, 593. 
sequential allocation, 401. 

Table-driven program, see Interpreter, 
Switching table. 

Tables, arrangement of, inside a computer, 
see Representation. 

Tables of numerical quantities, 54, 

66, 619-621. 

Tag field in tree node, 322, seeLTAG, RTAG. 

Tail inequalities, 104, 106-107. 

Tamaki, Jeanne Keiko 596. 

Tamari, Dov, 577. 
lattice, 577, 598. 

Tape, magnetic, 136-137, 463. 
paper, 136-137, 229, 231. 

Tarjan, Robert Endre, 243, 581. 

Taylor, Brook, series, 83, 102. 
with remainder, 117. 

Temme, Nicolaas Maria, 66, 121. 

Temporary storage: Part of memory used 
to hold a value for a comparatively 
short time while other values occupy 
the registers, 191. 

Termial function, 48, 51. 

Terminal node of a tree, 308, 318, 

352, 397, 597. 

Terminology, viii, 45, 240, 311, 362, 435. 

Ternary trees, 334, 401. 

Tetrad tiling, 383-385. 


Tetrahedral arrays, 300-301, 306. 

TgX, iv, xi, 193, 202, 611, 650. 

Theory of algorithms, 7, 9. 

Theory of automata, 230, 240, 463-464. 
Thiele, Thorvald Nicolai, 103. 

Thorelli, Lars-Erik, 603, 614. 

Thornton, Charles, 322, 459-460. 

Thread an unthreaded tree, 333. 

Thread links, 422. 

Threaded binary trees, 322, 331-332, 460. 
compared to unthreaded, 326. 
insertion into, 327, 332. 
list head in, 324, 334. 

Threaded trees, 335-336, 459. 

Three-address code, 337, 459. 

Tiling the plane, 383. 

Time taken by a program, see Execution 
time. 

Todd, John, 475. 

Todhunter, Isaac, 182. 

Tonge, Frederic McLanahan, Jr., 460. 
Top-down process, 309, 361. 

Top of stack, 241 242. 

Topological sorting, 261-271, 346, 376, 397. 
Torelli, Gabriele, 71, 488. 

Toscano, Letterio, 50. 

Toroidal tiling, 384. 

Total ordering, 270, see Linear ordering. 
Totient function <p(n), 42, 184. 

Trace routine, 192, 212-214, 230-231, 296. 
Traffic signals, 161-162. 

Trails, Eulerian, in a directed graph, 
374-376, 379-380, 584. 

Transfer instruction: A “jump” instruction. 
Transitive relation, 108, 261, 353, see 
Ordering. 

Transposing a rectangular matrix, 182. 
Transposing blocks of data, 185. 
Transpositions: Permutations that 
interchange two elements, 3, 182, 

274, 371. 

Traversal of binary trees, 319, 459-460. 
inorder, 320, 323. 
postorder, 331-332. 
preorder, 331-332. 

Traversal of trees, 336, 459-460. 

prepostorder, 568. 

Tree function T(z ), 395. 

Tree mappings, 390. 

Trees, 232, 308-423. 

binary, see Binary trees. * 

comparison of different types, 

308-309, 374. 

complete, 401 402, 405, 563. 
construction of, 340-341, 343, 428-429. 
copying of, 329-330, 333, 347. 
definition of, 308, 317, 363, 373. 
deletion from, 358. 

Dewey notation for, 313, 317, 

382-383, 460. 


INDEX AND GLOSSARY 649 


diagrams of, 309-315, 337, 346, 349, 460. 
disjoint, see Forest, 
embedding of, 348, 386. 
enumeration of, 386-399, 408. 
equivalent, 346. 
erasing of, 333. 
free, see Free trees, 
history, 406-407, 459-460. 
infinite, 317, 382. 
insertion into, 327, 332. 
labeled, enumeration of, 389, 407. 
linear ordering of, 346. 
linked allocation for, 334, 352-357. 
mathematical theory of, 362-408. 
ordered, 308, 374, 388-389, see Trees, 
oriented, see Oriented trees, 
quadruply linked, 357. 
representation of, 348-357, 359-362, 
459-460. 

right-threaded, 338, 380. 
sequential allocation for, 348-352, 
359-362, 433. 
similar, 346. 

f-ary, 334, 397, 401-402, 405, 593. 

ternary, 334, 401. 

threaded, 335-336, 459. 

traversal of, 336, 459-460. 

triply linked, 353, 359-360, 427-433. 

unordered, see Oriented trees. 

unrooted, 363, see Free trees. 

Triangular matrix, 300, 305. 

Triangulations of polygons, 407, 598-601. 
Tricomi, Francesco Giacomo Filippo, 

121 , 122 . 

Tridiagonal matrix, 307. 

Trigonometric functions, 44, 229, 471. 
Trilling, Laurent, 461. 

Triple order for a binary tree, 567-568. 
Triply linked tree, 353, 359-360, 427-433. 
Trit, 139. 

Tritter, Alan Levi, 576. 

Tucker, Alan Curtiss, 405. 

Turing, Alan Mathison, 18, 193, 229, 

230, 459. 

machines, 8, 230, 464. 

Tutte, William Thomas, 583. 

Twain, Mark (= Clemens, Samuel 
Langhorne), 54. 

Twigg, David William, 523. 

Two-line notation for permutations, 

164, 182. 

Two stacks, 246, 251, 253-254. 

Two-way linkage, 280-281, 411. 

Typewriter, 136-137, 231. 

Uhler, Horace Scudder, 480. 

Ullman, Jeffrey David, 560. 

Unary operator: A function of one 
variable, 337. 

UNDERFLOW, 245, 247, 259, 268-269, 274. 


Uniform distribution: A probability 

distribution in which every value is 
equally likely, 102, 253, 446. 

Unimodular matrices, 601. 

Uninitialized arrays, 307. 

Union-find algorithm, 354, 360. 

UNIVAC I computer, 151, 229, 231, 480. 
UNIVAC III computer, 124. 

UNIVAC SS80 computer, 124. 

UNIVAC 1107 computer, 124. 

UNIX operating system, 198. 

Unrooted trees, 363, see Free trees. 

Unusual correspondence between 
permutations, 178-179. 

Updates to memory, synchronous, 298. 
Uspensky, Vladimir Andreevich (ycneHCKHii, 
BjiaflHMup AHflpeeBHu), 464. 

van Aardenne-Ehrenfest, Tatyana, 375, 379. 
van der Waerden, Bartel Leendert, 

385-386, 587. 
van Leeuwen, Jan, 596. 
van Wijngaarden, Adriaan, 461. 
Vandermonde, Alexandre Theophile, 

59, 70, 71. 
matrix, 37-38, 475. 

Vardi, Ilan, 504. 

Variable: A quantity that may possess 
different values as a program is being 
executed, 3-4, 235. 
link or pointer, 235. 
node, 236. 

Variable-size nodes, 435-456. 

Variance of a probability distribution, 98. 
deduced from the generating function, 
100-103. 

Vauvenargues, Luc de Clapiers, Marquis 
de, xiv. 

Vectors, see Linear lists. 

Velthuis, Frans Jozef, 650. 

Vertex in a graph, 363, 372. 
isolated, 374. 

Victorius of Aquitania, 518. 

Virtual machine, 201. 

Visiting a node, 320. 

VLSI chips, 563. 
von Ettingshausen, Andreas, 54. 
von Neumann, John (= Margittai Neumann 
Janos), 18, 229, 231, 457. 
von Segner, Johann Andreas, 407, 536. 
von Staudt, Karl Georg Christian, 406. 

W-value in MIXAL, 154-155. 

Wadler, Philip Lee, 605. 

Waerden, Bartel Leendert van der, 

385-386, 587. 

Wait list, see Agenda. 

Waite, William McCastline, 418, 422, 614. 
Wall, Hubert Stanley, 481. 

Wallis, John, 22, 52. 
product for 7r, 52, 116. 


650 INDEX AND GLOSSARY 


Wang, Hao (3E?n), 383-384. 

Waring, Edward, 78, 472. 

Warren, Don Wyman, 359. 

Watanabe, Masatoshi 650. 

Watson, Dan Caldwell, 251. 

Watson, George Neville, 507. 
lemma, 123. 

Watson, Henry William, 383. 

Weakest precondition, 17. 

Weber, Helmut, 461. 

Webster, Noah, dictionary, 1, 216. 
Wedderburn, Joseph Henry Maclagan, 589. 
Wegbreit, Eliot Ben, 614. 

Wegner, Peter (= Weiden, Puttilo Leonovich 
= BcvLaeii. I Ivtthjio JleoHOBnu), 306. 
Weierstrass (= Weierstrafi), Karl Theodor 
Wilhelm, 382. 

Weighted path length, 402-405. 

Weiland, Richard Joel, 608. 

Weizenbaum, Joseph, 414, 459-461. 
Well-ordering, 20, 334. 

Wheeler, David John, 229, 230, 457. 
Whinihan, Michael James, 86. 

Whirlwind I computer, 230. 

Whitworth, William Allen, 182. 

Wijngaarden, Adriaan van, 461. 

Wilde, Oscar Fingal O’Flahertie Wills, 422. 
Wiles, Andrew John, 466. 

Wilf, Herbert Saul, 65, 66, 93, 94, 484. 
Wilkes, Maurice Vincent, 229, 230, 457. 
Wilson, John, theorem, 51. 

Wilson, Paul Robinson, 452. 

Windley, Peter F., 523. 

Windsor, House of, 310. 

Winkler, Phyllis Astrid Benson, xi. 

Wirth, Niklaus Emil, 191, 461. 

Wise, David Stephen, 251, 421, 434, 605. 
Wiseman, Neil Ernest, 421. 

Wolman, Eric, 453. 

Wolontis, Vidar Michael, 230. 

Woods Berners-Lee, Mary Lee, 230. 
Woodward, Philip Mayne, 460. 


Word: An addressable unit of computer 
memory, 125-127. 

Word size, for MIX: The number of 

different values that might be stored 
in five bytes. 

Wordsworth, William, 139. 

Worst-fit method of storage allocation, 

453, 608. 

Wortman, David Barkley, 230. 

Wrench, John William, Jr., 480, 621. 

Wright, Edward Maitland, 492, 520. 

Wright, Jesse Bowdle, 359. 

Writing: Doing output, 215. 

Writing large programs, 191-193. 

Wyman, Max, 66. 

Wythoff (= Wijthoff), Willem Abraham, 80. 

X-l computer, 231. 

X-register of MIX, 125. 

XDS 920 computer, 124. 

X0R (exclusive or), 455. 

Yang Hui (ffi'IS), 53. 

Yao, Andrew Chi-Chih (ttKIU), 543. 

Yngve, Victor Huse, 461. 

Yo-yo list, 240. 

Yoder, Michael Franz, 479. 

Young, David Monaghan, Jr., 586. 

z, for complex numbers, 21, 108. 

Zabell, Sandy Lew, 491. 

Zaks, Shinuel (Dpt pNlOW), 598. 

Zave, Derek Alan, 91, 614. 

Zeckendorf, Edouard, 495. 

Zeilberger, Doron inn), 65. 

Zemanek (= Zemanek), Heinz, 1. 

Zeta function f(s,x), 44, 76, 504. 

Zhang, Linbo (®#lk), 650. 

Zimmerman, Seth, 407. 

Zorn, Max, lemma, 547. 

Zuse, Konrad, 457. 


We must not . . . think that computation, 
that is ratiocination, 
has place only in numbers. 
— THOMAS HOBBES, Elementary Philosophy (1656) 
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(Reading, Mass.: Addison- Wesley, 1986), Volumes A-E. The illustrations were produced with 
John Hobby’s METHPOST system. Some names in the index were typeset with additional 
fonts developed by Yannis Haralambous (Greek, Hebrew, Arabic), Olga G. Lapko (Cyrillic), 
Frans J. Velthuis (Devanagari), Masatoshi Watanabe (Japanese), and Linbo Zhang (Chinese). 






Character code: 


00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 

uABCDEFGHIAJKLMNOPqREIISTU 


00 

1 

01 

2 

02 

2 

03 

10 

No operation 

N0P(0) 

rA 4- rA + V 

ADD(0 : 5) 
FADD(6) 

rA 4- rA - V 

SUB (0:5) 
FSUB(6) 

rAX 4— rA x V 

MUL(0:5) 

FMUL(6) 

08 

2 

09 

2 

10 

2 

11 

2 

rA 4- V 

LDA(0 : 5) 

rll 4- V 

LD1 (0 : 5) 

rI2 4- V 

LD2(0:5) 

rI3 4- V 

LD3(0 : 5) 

16 

2 

17 

2 

18 

2 

19 

2 

rA 4- -V 

LDAN (0 : 5) 

rll 4- -V 

LD1N (0 : 5) 

rI2 4- -V 

LD2N(0 : 5) 

rI3 4- -V 

LD3N (0 : 5) 

24 2 

25 

2 

26 

2 

27 

2 

M(F) 4- rA 

STA(0:5) 

M(F) 4- rll 

ST1 (0 : 5) 

M(F) 4- rI2 

ST2(0:5) 

M(F) 4- rI3 

ST3(0 : 5) 

32 

2 

33 2 

34 

1 

35 

1 + T 

M(F) 4- rJ 

STJ (0 : 2) 

M(F) 4- 0 

STZ(0:5) 

Unit F busy? 

JBUS(0) 

Control, unit F 

I0C(0) 

40 

1 

41 

1 

42 

1 

43 

1 

rA : 0, jump 

JA [+] 

rll : 0, jump 

J1C+] 

rI2 : 0, jump 

J2 [+] 

rI3 : 0, jump 

J3 [+] 

48 1 

49 1 

50 

1 

51 1 

rA 4— [rA]? ± M 

INCA(O) DECA(l) 
ENTAC2) ENNAC3) 

rll 4— [rll]? ± M 

INCl(O) DECl(l) 
ENT1(2) ENN1C3) 

rI2 4- [rI2]? ± M 

INC2(0) DEC2C1) 
ENT2(2) ENN2(3) 

rI3 4- [rI3] ? ± M 

INC3(0) DEC3C1) 
ENT3(2) ENN3(3) 

56 2 

57 2 

58 2 

59 2 

Cl 4- rA(F) : V 

CMPA(0:5) 

FCMP(6) 

Cl 4- rll(F) : V 

CMP1 (0 : 5) 

Cl 4- rI2(F) : V 

CMP2 (0 : 5) 

Cl 4- rI3(F) : V 

CMP3(0 : 5) 


C = operation code, (5 : 5) field of instruction 
F = op variant, (4 : 4) field of instruction 
M = address of instruction after indexing 
V = M(F) = contents of F field of location M 
OP = symbolic name for operation 
(F) = normal F setting 

t = execution time; T = interlock time 


General form: 




25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 

VWXYZ0123456789 . , ()+-*/ 


48 49 50 51 52 53 54 55 

= $ < > a ; : ' 


04 


12 


rA 4 - rAX/V 
rX 4 — remainder 
DIV(0:5) 
FDIV(6) 


12 


2 


rI4 4- V 
LD4(0 : 5) 


20 


rI4 4- -V 
LD4N (0 : 5) 


28 


2 


M(F) 4 — rI4 
ST4(0 : 5) 


36 


1 + T 


Input, unit F 
IN (0) 


44 


1 


rI4 : 0, jump 
J4[+] 


05 


10 


Special 
NUM(O) 
CHAR ( 1 ) 
HLT(2) 


13 


rI5 4- V 
LD5(0 : 5) 


21 


rI5 4- -V 
LD5N(0:5) 


29 


2 


M(F) 4— rI5 
ST5(0:5) 


37 


1 + T 


Output, unit F 
OUT(O) 


45 


1 


rI5 : 0, jump 
J5 [+] 


06 


2 


Shift M bytes 
SLA(O) SRA(l) 
SLAX(2) SRAXC3) 
SLC(4) SRC (5) 


14 


2 


rI6 4 — V 
LD6(0:5) 


22 


rI6 4- -V 
LD6N(0:5) 


30 


2 


M(F) 4— rI6 
ST6(0:5) 


38 


1 


Unit F ready? 
JRED(O) 


46 


rI6 : 0, jump 
J6[+] 


07 


1 + 2 F 


Move F words 
from M to rll 

MOVE ( 1 ) 


15 


rX 4— V 
LDX(0 : 5) 


23 


rX 4 V 

LDXN(0:5) 


31 


2 


M(F) 4 - rX 
STX(0 : 5) 


39 


1 


Jumps 

JMP(O) JSJ(l) 
JOV (2) JN0VC3) 
also [*] below 


47 


1 


rX : 0, jump 
JX[+] 


52 


53 


1 


54 


rI4 4- [rI4] ? ± M 

INC4(0) DEC4(1) 
ENT4C2) ENN4(3) 


rI5 4- [rI5] ? ± M 

INC5(0) DEC5C1) 
ENT5(2) ENN5(3) 


55 


rI6 4- [rI6] ? ± M 

INC6(0) DEC6(1) 
ENT6(2) ENN6(3) 


1 


rX 4- [rX]? ± M 

INCX(O) DECX(l) 
ENTX(2) ENNX(3) 


60 


61 


2 


62 


Cl 4- rI4(F) : V 
CMP4(0 : 5) 


Cl 4- rI5(F) : V 
CMP5(0:5) 


2 


63 


Cl 4- rI6(F) : V 
CMP6(0:5) 


2 


Cl 4- rX(F) : V 
CMPX(0 : 5) 


rA = register A 
rX = register X 

rAX = registers A and X as one 
rli = index register i, 1 < i < 6 
rJ = register J 
Cl = comparison indicator 


[*] : 


[+] : 

JL(4) 

< 

N(0) 

JE(5) 


Z(l) 

JG(6) 

■ 

P(2) 

JGE(7) 

> 

NN (3) 

JNE(8) 


NZ(4) 

JLE(9) 

< 

NP(5) 


25 26 27 28 29 30 31 32 33 34 35 36 37 38 

VWXYZ012345678 


04 

12 

05 

1 

rA 4- rAX/V 
rX 4— remainder 
DIV(0:5) 
FDIV(6) 

Special 
NUM(O) 
CHAR( 1) 
HLTC2) 

12 

2 

13 

A 

rI4 •<- V 

rI5+- V 

LD4(0:5) 

LD5(0: 

5) 

20 

2 

21 

1 

A 

rI4 < — 

-V 

rI5 i — 

-V 

LD4N(0:5) 

LD5N(0:5) 

28 

2 

29 

t 

A 

M(F) «— rI4 

M(F) 4— rI5 

ST4(0: 

5) 

ST5(0: 

5) 

36 

1 + T 

37 

14 

Input, unit F 

Output, unit F 

IN(0) 

OUT(O) 

44 

1 

45 

J 

rI4 : 0, jump 

rI5 : 0, jump 

J4[+] 


J5[+] 


52 

1 

53 

1 

rI4 4- [rI4] ? ± M 

INC4(0) DEC4(1) 
ENT4(2) ENN4(3) 

rI5 <— [rI5] ? ± A 

INC5(0) DEC50 
ENT5(2) ENN5 (c 

60 

2 

61 

2 

Cl 4- rI4(F) : V 

Cl <- rI5(F) : \ 

CMP4(0 : 

5) 

CMP5(0: 

5) 


o: 

rA = register A JL 

rX = register X JEi 

rAX = registers A and X as one JG 

rli = index register *, 1 < i < 6 JGE 
rJ = register J JNEi 

Cl = comparison indicator JLEi 
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